Field programmable gate array with internal phase-locked loop

ABSTRACT

The present invention relates to a field programmable gate array system that provides phase control with minimal latency.

REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of and priority to U.S. ProvisionalPatent Application No. 62/808,404, filed on Feb. 21, 2019 and entitledFIELD PROGRAMMABLE GATE ARRAY WITH EXTERNAL PHASE-LOCKED LOOP, theentire content of which is hereby incorporated by reference herein.

FIELD

The present invention generally relates to a field programmable gatearray system. In embodiments, the present invention generally relates toa field programmable gate array and an external or internal phasecontroller providing phase matching between a receiver clock and atransmitter clock used in the field programmable gate array.

BACKGROUND

Field programmable gate arrays (FPGAs) may be used in applications thatrequire fast processing since FPGAs allow for all computations to occuron a single chip that has massive fine-grained parallelism. For example,FPGAs are used in the financial industry in high frequency trading wherethe rapid processing of the FPGA is desired. One technological problemwith FPGAs is that there is a need to synchronize receiving side andtransmitting side clock signals within the FPGA.

The prior art sought to address this problem by including a clock domaincrossing circuit in the FPGA. However, these circuits inherently add adelay to the processing that takes place in the FPGA, which is notdesirable since high frequency trading may include timestamps that areaccurate to the microsecond such that even small delays may present alarge problem.

Synchronous Ethernet systems were designed with the similar goal ofsynchronizing a transmitter to a receiver. However, since phasealignment in synchronous Ethernet is not necessary, synchronous EthernetFPGA systems usually do not phase-align the receiver and transmittersides of a link. The received clock is output from the transceiver to anexternal clock generation circuit to be used as its frequency reference.As a result, there is no measurement of the internal transmit clock atall by the Ethernet FPGA system.

While a synchronous Ethernet system with an internal PLL compares thetransmit clock to the receive clock, its phase-frequency detector in theinternal PLL is configured to measure accurately only the frequency butdoes not have to measure the phase of the two clocks accurately. Inaddition, synchronized protocols of the synchronous Ethernet system usesimpler control loops that do not have to compensate for a lot of wanderor jitter. Accordingly, the phase of the clocks is not treated asimportant as frequency by the synchronous Ethernet FPGA system. Based onthese reasons, synchronous Ethernet FPGA systems do not provide adesirable solution to a technological problem of synchronizing receivingside and transmitting side clock signals within the FPGA.

Accordingly, a technical problem is presented in FPGAs in that phasesynchronization between the receiver side clock and the transmitter sideclock will introduce unwanted latency that results in delay ofprocessing. Accordingly, it would be beneficial to provide an FPGAsystem that avoids these problems and provides sub-microsecondprocessing with throughput of at least 10 Gbps. In embodiments, thesub-microsecond processing throughput may be: 10-25 Gbps, 22-33 Gbps,33-45 Gbps, 45-60 Gbps, 60-80 Gbps, 80-120 Gbps, to name a few.

SUMMARY

An object of the present invention is to address technologicalchallenges that currently exist in phase matching receiver side andtransmitter side clocks of a FPGA without introducing unnecessary delayin processing.

This and other objects shall be addressed by embodiments of the presentinvention as set forth herein.

The present invention generally relates to a field programmable gatearray system and a method for processing a data stream using a fieldprogrammable gate array system. In embodiments, the present inventiongenerally relates to a field programmable gate array and an external orinternal phase controller providing phase matching between a receiverclock and a transmitter clock used in the field programmable gate array.

More particularly, the present invention relates to a field programmablegate array system includes (a) a field programmable gate arraycomprising (1) a first interface including: (A) a first reference clockpin, wherein said first reference clock pin is configured to receive afirst clock signal having a first frequency and a first phase; (B) asecond reference clock pin, wherein said second reference clock pin isconfigured to receive a second clock signal having a second frequencyand a second phase; (C) a first plurality of data pins, wherein saidfirst plurality of data pins is configured to receive a first serialdata stream; (D) a second plurality of data pins, wherein said secondplurality of data pins is configured to transmit a second serial datastream; (2) a deserializer operationally connected to: (x) the firstreference clock pin to receive as a first input the first clock signaland (y) the first plurality of data pins to receive as a second inputthe first serial data stream, and wherein the deserializer is configuredto: (A) convert the first serial data stream into a first plurality ofparallel data streams having a first amount of data streams, and (B)generate a first receiver side clock signal based on the first clocksignal, wherein the first receiver side clock signal has a thirdfrequency and a third phase; and (C) transmit the first plurality ofparallel data streams and the first receiver side clock signal withinthe field programmable gate array; (3) computational circuitryoperationally connected to the deserializer to receive the firstplurality of parallel data streams and the first receiver side clocksignal, wherein the computational circuitry is configured to perform afirst set of operations on the first plurality of parallel data streamsto generate a second plurality of parallel processed data streams havinga second amount of data streams; (4) a serializer operationallyconnected to: (x) the second reference clock pin to receive as a thirdinput a first wire rate clock signal based on the second clock signal,wherein the first wire rate clock signal has a fourth frequency and afourth phase; (y) the second plurality of data pins to transmit as afirst output the second serial data stream; (z) the computationalcircuitry, wherein the serializer receives the second plurality ofparallel processed data streams from the computational circuitry and theserializer transmits to the computational circuitry a first transmitterside clock signal including a fifth frequency and a fifth phase; andwherein the serializer is configured to: (A) convert the secondplurality of parallel processed data streams into the second serial datastream; (B) generate the first transmitter side clock signal based onthe first wire rate signal, wherein the first transmitter side clocksignal has the fifth frequency and the fifth phase, wherein the fifthfrequency is different than and less than the fourth frequency; and (C)transmit the second serial data stream to the second plurality of datapins for transmission off the field programmable gate array; (5) asecond interface including: (A) a first clock output pin configured totransmit the first receiver side clock signal, wherein the first clockoutput pin is operationally connected to the deserializer; and (B) asecond clock output pin configured to transmit the first transmitterside clock signal, wherein the second clock output pin is operationallyconnected to the serializer; and (b) a phase control circuit, providedat least partially on the field programmable gate array, wherein thephase control circuit includes: (1) a phase detector operationallyconnected to the first clock output pin and the second clock output pinof the second interface of the field programmable gate array, andwherein the phase detector is configured to compare the third phase ofthe first receiver side clock signal to the fifth phase of thetransmitter side clock signal and to generate a phase differenceindicator signal based on a difference between the third phase of thefirst receiver side clock signal and the fifth phase of the transmitterside clock signal; (2) a phase controller operationally connected to thephase detector and configured to receive the phase difference indicatorsignal, and wherein the phase controller is configured to determineadjustment information based on the phase difference indicator signal;and (3) an adjustable oscillator operationally connected to the phasecontroller and configured to receive the adjustment information as wellas operationally connected to the second reference clock pin of thefirst interface of the field programmable gate array, wherein theadjustable oscillator is configured to generate the second clock signalincluding the second frequency and the second phase based on theadjustment information and transmit the second clock signal to thesecond reference clock pin of the first interface of the fieldprogrammable gate array; wherein the transmitter side clock signal andthe first receiver side clock signal are phase aligned so that there isa fixed phase difference between the third phase and the fifth phase.

In embodiments, the third frequency corresponds to the first frequency.

In embodiments, the third phase is not aligned with the first phase.

In embodiments, the third frequency is different than the firstfrequency.

In embodiments, the first set of operations does not include clockdomain crossing operations that delays processing of the first set ofparallel data streams. In embodiments, the adjustment information isused to do at least one of the following (i) set a bias to anoscillator; (ii) to set a divider ratio; and (iii) to set a delay.

In embodiments, the first set of operations includes an arithmeticoperation. In embodiments, the first set of operations includes alogical operation. In embodiments, the first set of operations includesa pipeline operation. In embodiments, the first set of operationsincludes a memory access operation.

In embodiments, the first amount of data streams is the same as thesecond amount of data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to eight (8) data streams. In embodiments, thefirst amount of data streams and the second amount of data streams areequal to ten (10) data streams. In embodiments, the first amount of datastreams and the second amount of data streams are equal to sixteen (16)data streams. In embodiments, the first amount of data streams and thesecond amount of data streams are equal to twenty (20) data streams. Inembodiments, the first amount of data streams and the second amount ofdata streams are equal to thirty-two (32) data streams. In embodiments,the first amount of data streams and the second amount of data streamsare equal to forty (40) data streams. In embodiments, the first amountof data streams and the second amount of data streams are equal tosixty-four (64) data streams. In embodiments, the first amount of datastreams and the second amount of data streams are equal to eighty (80)data streams. In embodiments, the first amount of data streams and thesecond amount of data streams are equal to one hundred twenty-eight(128) data streams. In embodiments, the first amount of data streams andthe second amount of data streams are equal to one hundred sixty (160)data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to two (2) to the power of N, where N is aninteger. In embodiments, the first amount of data streams and the secondamount of data streams are equal to ten (10) times two (2) to the powerof N, where N is an integer. In embodiments the first amount of datastreams and the second amount of data streams are equal to thirty-three(33) times two (2) to the power of N, where N is an integer.

In embodiments, the field programmable gate array further includes aphase lock loop configured to: receive the second clock signal, as afourth input, to generate the first wire rate clock signal; and (B)transmit the first wire rate clock signal as the third input, to theserializer.

In embodiments, the fourth frequency is equal to x times the fifthfrequency, where x is the second amount of data streams.

In embodiments, the fourth frequency is equal to x times the fifthfrequency, where x is half of the second amount of data streams.

In embodiments, the fourth frequency is a rational multiple of thesecond frequency.

In embodiments, the phase difference indicator signal is a pulse signal.

In embodiments, the adjustment information indicates a desired phase. Inembodiments, the adjustment information indicates a desired frequency.In embodiments, the adjustment information indicates a change in phase.In embodiments, the adjustment information indicates a change infrequency. In embodiments, adjustment information comprises a voltage.In embodiments, adjustment information comprises a digital transmission.In embodiments adjustment information further comprises a digitaltransmission.

In some embodiments, the first serial data stream comprises market data,the second serial data stream comprises order entry data, and the firstoperation comprises a trading algorithm.

In embodiments, the trading algorithm includes the steps of: (a) parsingmarket data; (b) performing mathematical operations at a portion of themarket data; and (c) generating order packets using at least an outputof (b).

In embodiments, the first serial data stream includes market data andthe second serial stream includes trading data.

In embodiments, the field programmable gate array includes a transceiverphase locked loop operatively connected between the to the serializerand the first reference clock pin.

In embodiments, the transceiver phase locked loop operatively connectedbetween the serializer and the first reference clock pin includes asecond adjustable oscillator to provide the first wire rate signal.

In embodiments, a second adjustable oscillator is provided outside ofthe transceiver phase locked loop and operatively connected to theserializer.

In embodiments, the second adjustable oscillator is programmable with adesired delay.

In embodiments, the desired delay is a constant value

In embodiments, the constant value is zero.

In embodiments, the adjustment information is provided to the secondadjustable oscillator to provide the first wire rate signal.

The present invention also relates to a field programmable gate arraysystem comprising (a) a field programmable gate array comprising (1) afirst interface comprising (A) a first reference clock pin, wherein saidfirst reference clock pin is configured to receive a first clock signalhaving a first frequency and a first phase; (B) a second reference clockpin, wherein said second reference clock pin is configured to receive asecond clock signal having a second frequency and a second phase; (C) afirst plurality of data pins, wherein said first plurality of data pinsis configured to receive a first serial data stream; (D) a secondplurality of data pins, wherein said second plurality of data pins isconfigured to transmit a second serial data stream; (2) a deserializeroperationally connected to (A) the first reference clock pin to receiveas a first input the first clock signal and (B) the first plurality ofdata pins to receive as a second input the first serial data stream, andwherein the deserializer is configured to (A) convert the first serialdata stream into a first plurality of parallel data streams having afirst amount of data streams, and (B) generate a first receiver sideclock signal based on the first clock signal, wherein the first receiverside clock signal has a third frequency and a third phase; and (C)transmit the first plurality of parallel data streams and the firstreceiver side clock signal within the field programmable gate array; (3)computational circuitry operationally connected to the deserializer toreceive the first plurality of parallel data streams and the firstreceiver side clock signal, wherein the computational circuitry isconfigured to perform a first set of operations on the first pluralityof parallel data streams to generate a second plurality of parallelprocessed data streams having a second amount of data streams; (4) aserializer operationally connected to (A) a transceiver phase lock loopto receive as a third input a first wire rate clock signal based on thesecond clock signal, wherein the first wire rate clock signal has afourth frequency and a fourth phase; (B) the second plurality of datapins to transmit as a first output the second serial data stream; (C)the computational circuitry, wherein the serializer receives the secondplurality of parallel processed data streams from the computationalcircuitry and the serializer transmits to the computational circuitry afirst transmitter side clock signal including a fifth frequency and afifth phase; and wherein the serializer is configured to (A) convert thesecond plurality of parallel processed data streams into the secondserial data stream; (B) generate the first transmitter side clock signalbased on the first wire rate signal, wherein the first transmitter sideclock signal has the fifth frequency and the fifth phase, wherein thefifth frequency is different than and less than the fourth frequency;and (C) transmit the second serial data stream to the second pluralityof data pins for transmission off the field programmable gate array; (5)a second interface comprising (A) a first clock output pin configured totransmit the first receiver side clock signal, wherein the first clockoutput pin is operationally connected to the deserializer; and (B) asecond clock output pin configured to transmit the first transmitterside clock signal, wherein the second clock output pin is operationallyconnected to the serializer; and (b) a phase control circuit, providedat least partially on the field programmable gate array, wherein thephase control circuit comprises (1) a phase detector operationallyconnected to the first clock output pin and the second clock output pinof the second interface of the field programmable gate array, andwherein the phase detector is configured to compare the third phase ofthe first receiver side clock signal to the fifth phase of thetransmitter side clock signal and to generate a phase differenceindicator signal based on a difference between the third phase of thefirst receiver side clock signal and the fifth phase of the transmitterside clock signal; (2) a phase controller operationally connected to thephase detector and configured to receive the phase difference indicatorsignal, and wherein the phase controller is configured to determineadjustment information based on the phase difference indicator signal;and (3) an adjustable oscillator operationally connected to the phasecontroller and configured to receive the adjustment information as wellas operationally connected to the second reference clock pin of thefirst interface of the field programmable gate array, wherein theadjustable oscillator is configured to generate the second clock signalincluding the second frequency and the second phase based on theadjustment information and transmit the second clock signal to thesecond reference clock pin of the first interface of the fieldprogrammable gate array, wherein the transmitter side clock signal andthe first receiver side clock signal are phase aligned so that there isa fixed phase difference between the third phase and the fifth phase.

In embodiments, the third frequency corresponds to the first frequency.

In embodiments, the third phase is not aligned with the first phase.

In embodiments, the third frequency is different than the firstfrequency.

In embodiments, the first set of operations does not include clockdomain crossing operations that delays processing of the first set ofparallel data streams.

In embodiments, the first set of operations includes at least one of thefollowing: (i) an arithmetic operation; (ii) a logical operation; (iii)a pipeline operation; and (iv) a memory access operation.

In embodiments, the first amount of data streams is the same as thesecond amount of data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to one of the following: (i) eight (8) datastreams; (ii) ten (10) data streams; (iii) sixteen (16) data streams;(iv) twenty (20) data streams; (v) thirty-two (32) data streams; (vi)forty (40) data streams; (vii) sixty-four (64) data streams; (viii)eighty (80) data streams; (ix) one hundred twenty-eight (128) datastreams; and (x) one hundred sixty (160) data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to two (2) to the power of N, where N is aninteger.

In embodiments, the first amount of data streams and the second amountof data streams are equal to ten (10) times two (2) to the power of N,where N is an integer.

In embodiments, the first amount of data streams and the second amountof data streams are equal to thirty-three (33) times two (2) to thepower of N, where N is an integer.

In embodiments, the transceiver phase lock loop is operationallyconnected to the second reference clock pin and is configured to: (A)receive the second clock signal, as a fourth input, to generate thefirst wire rate clock signal; and (B) transmit the first wire rate clocksignal as the third input, to the serializer.

In embodiments, the fourth frequency is equal to x times the fifthfrequency, where x is the second amount of data streams.

In embodiments, the fourth frequency is equal to x times the fifthfrequency, where x is half of the second amount of data streams.

In embodiments, the fourth frequency is a rational multiple of thesecond frequency.

In embodiments, the phase difference indicator signal is a pulse signal.

In embodiments, adjustment information indicates at least one of thefollowing:

-   -   (A) a desired phase; (B) a desired frequency; (C) a change in        phase; and (D) a change in frequency.

In embodiments, adjustment information comprises a voltage.

In embodiments, adjustment information comprise a digital transmission.

In embodiments, the adjustment information is used to do at least one ofthe following: (i) set a bias to an oscillator; (ii) set a dividerratio; and (iii) set a delay.

In embodiments, the first serial data stream comprises market data, thesecond serial data stream comprises order entry data, and the firstoperation comprises a trading algorithm.

In embodiments, the first serial data stream includes market data andthe second serial data stream includes trading data.

In embodiments, the field programmable gate array system furthercomprises (a) a fourth plurality of data pins of the first interface,wherein the fourth plurality of data pins is configured to transmit afourth serial data stream; (b) a second deserializer operationallyconnected to (i) the first reference clock pin to receive the firstclock signal as a third input to the second deserializer, wherein thesecond deserializer is configured to: (A) receive a third serial datastream; (B) convert the third serial data stream into a third pluralityof parallel data streams having a third amount of data streams; (C)generate a second receiver side clock signal based on the first clocksignal, wherein the second receiver side clock signal has a sixthfrequency and a sixth phase; and (D) transmit within the fieldprogrammable gate array to the computational circuitry the thirdplurality of parallel data streams and the second receiver side clocksignal; (ii) the computational circuitry further configured to (A)receive the third plurality of parallel data streams and the secondreceiver side clock signal; and (B) generate a fourth plurality ofparallel processed data streams having a fourth amount of data streams;(c) a second serializer operationally connected to: (i) the transceiverphase lock loop to receive as a fifth input the first wire rate clocksignal based on the second clock signal; and (ii) the fourth pluralityof data pins to transmit as a second output the fourth serial datastream, wherein the second serializer is configured to (A) receive thefourth plurality of parallel processed data streams from thecomputational circuitry; (B) convert the fourth plurality of parallelprocessed data streams into the fourth serial data stream; (C) transmitthe fourth serial data stream to the fourth plurality of data pins fortransmission off the field programmable gate array; (d) a third clockoutput pin of the second interface configured to transmit the secondreceiver side clock signal, wherein the third clock output pin isoperationally connected to the second deserializer; and (e) a secondphase control circuit, provided at least partially on the fieldprogrammable gate array, wherein the phase control circuit comprises:(i) a second phase detector operationally connected to the first clockoutput pin and the third clock output pin of the second interface of thefield programmable gate array, wherein the second phase detector isconfigured to (A) compare the third phase of the first receiver sideclock signal to the sixth phase of the second receiver side clocksignal; and (B) generate a second phase difference indicator signalbased on a difference between the third phase of the first receiver sideclock signal and the sixth phase of the second receiver side clocksignal; (ii) a second phase controller operationally connected to thesecond phase detector and configured to receive the second phasedifference indicator signal, wherein the second phase controller isconfigured to determine second adjustment information based on thesecond phase difference indicator signal; and (iii) a delay elementoperationally connected to the second phase controller and to the seconddeserializer, wherein the second phase controller sends to the delayelement the second adjustment information, wherein the delay element isconfigured to (A) delay the third serial data stream based on the secondadjustment information, and (B) transmit the third serial data stream tothe second deserializer, wherein the first receiver side clock signaland the second receiver side clock signal are phase aligned so thatthere is a fixed phase difference between the third phase and the sixthphase.

In embodiments, the field programmable gate array system furthercomprises a third plurality of data pins of the first interface, whereinthe delay element is placed on the field programmable gate array andoperationally connected to the third plurality of data pins and whereinthe delay element is further configured to (A) receive the third serialdata stream via the third plurality of data pins, and (B) transmit thethird serial data stream to the second deserializer.

In embodiments, the field programmable gate array system furthercomprises a third plurality of data pins of the first interface, wherein(A) the delay element is placed outside of the field programmable gatearray, (B) the delay element and the second deserializer areoperationally connected to the third plurality of data pins, and (C) thethird plurality of data pins is further configured to transmit the thirdserial data stream from the delay element to the second deserializer.

The present invention also relates to a method for processing a firstserial data stream comprising market data, using a field programmablegate array system, to generate a second serial data stream comprisingorder entry data, wherein the method comprises the steps of (a)receiving, by a first plurality of data pins of a first interface of afield programmable gate array in the field programmable gate arraysystem, the first serial data stream; (b) receiving, by a firstreference clock pin in the first interface, a first clock signal havinga first frequency and a first phase; (c) transmitting, from the firstplurality of data pins of the first interface to a deserializer in thefield programmable gate array, the first serial data stream; (d)transmitting, from the first reference clock pin in the first interfaceto the deserializer, the first clock signal; (e) generating, by thedeserializer, a first receiver side clock signal having a secondfrequency and a second phase, based on the first clock signal; (f)converting, by the deserializer, the first serial data stream into afirst plurality of parallel data streams; (g) transmitting, from thedeserializer to computational circuitry in the field programmable gatearray, the first receiver side clock signal; (h) transmitting, from thedeserializer to the computational circuitry, the first plurality of datastreams comprising a first plurality of data items and a first amount ofdata streams; (i) transmitting, from the deserializer via a first clockoutput pin on a second interface of the field programmable gate array toa phase detector that is part of the field programmable gate arraysystem and not part of the field programmable gate array, the firstreceiver side clock signal; (j) generating, using the field programmablegate array system, a first transmitter side clock signal having a thirdfrequency and a third phase, by performing the following steps until afirst output of the phase detector is below a first threshold level: (i)generating, by an adjustable oscillator in the field programmable gatearray system, a second clock signal having a fourth frequency and afourth phase; (ii) generating a first wire rate clock signal having afifth frequency and a fifth phase based on the second clock signal;(iii) generating, by a serializer in the field programmable gate array,an interim transmitter side clock signal having a sixth frequency and asixth phase; (iv) transmitting, from the serializer via a second clockoutput pin of the second interface to the phase detector, the interimtransmitter side clock signal; (v) generating, by the phase detector,the first output based on a comparison of the first receiver side clocksignal and the interim transmitter side clock signal; (vi) transmitting,from the phase detector to a phase controller of the field programmablegate array system which is placed within the field programmable gatearray, the first output; (vii) determining, by the phase controller,interim adjustment information based on the first output; and (viii)transmitting, from the phase controller to the adjustable oscillator,the interim adjustment information, wherein, the adjustable oscillatoradjusts the second clock signal based on the interim adjustmentinformation and steps (i) through (viii) are repeated until the firstoutput of the phase detector is below the first threshold level; (k)transmitting, from the serializer to the computational circuitry, firsttransmitter side clock signal; (1) performing, by the computationalcircuitry, a first set of operations on at least a portion of the firstplurality of data items to generate a second plurality of parallel datastreams comprising a second amount of data streams; (m) transmitting,from the computational circuitry to the serializer, the second pluralityof parallel data streams; (n) converting, by the serializer, the secondplurality of parallel data streams into the second serial data stream;and (o) transmitting, from the serializer off the field gateprogrammable array system via a second plurality of data pins of thefirst interface, the second serial data stream.

In embodiments, step (a) is performed before step (b).

In embodiments, step (b) is performed before step (a).

In embodiments, step (a) and step (b) are performed contemporaneously.

In embodiments, step (c) is performed before step (d).

In embodiments, step (d) is performed before step (c).

In embodiments, step (c) and step (d) are performed contemporaneously.

In embodiments, step (e) is performed before step (f).

In embodiments, step (f) is performed before step (e).

In embodiments, step (e) and step (f) are performed contemporaneously.

In embodiments, the second frequency corresponds to the first frequency.

In embodiments, the second phase is not aligned with the first phase.

In embodiments, the second frequency is different from the firstfrequency.

In embodiments, step (g) is performed before step (h).

In embodiments, step (h) is performed before step (g).

In embodiments, step (g) and step (h) are performed contemporaneously.

In embodiments, the step (i) comprises transmitting, from thedeserializer to the phase detector, the first receiver side clock signalvia a first zero delay buffer phase lock loop of the field programmablegate array before transmitting the first receiver side clock signal viathe first clock output pin of the second interface.

In embodiments, the third frequency corresponds to the second frequency.

In embodiments, the third phase is aligned with the second phase.

In embodiments, a difference between the third phase and the secondphase is less than a second threshold level

In embodiments, the sixth frequency corresponds to the third frequency.

In embodiments, the sixth phase is not aligned with the third phaseprior to the first output of the phase detector being below the firstthreshold level.

In embodiments, the step (j)(iv) comprises transmitting, from theserializer to the phase detector, the interim transmitter side clocksignal via a second zero delay buffer phase lock loop of the fieldprogrammable gate array before transmitting the interim transmitter sideclock signal via the second clock output pin of the second interface.

In embodiments, the first set of operations does not include clockdomain crossing operations that delays processing of the first set ofparallel data streams.

In embodiments, the first amount of data streams is the same as thesecond amount of data streams.

In embodiments, the first amount of data streams and the second amountof data streams is one of the following: (i) eight (8) data streams;(ii) ten (10) data streams; (iii) sixteen (16) data streams; (iv) twenty(20) data streams; (v) thirty-two (32) data streams; (vi) forty (40)data streams; (vii) sixty-four (64) data streams; and (viii) eighty (80)data streams; (ix) one hundred twenty-eight (128) data streams; and (x)one hundred sixty (160) data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to two (2) to the power of N, where N is aninteger.

In embodiments, the first amount of data streams and the second amountof data streams are equal to ten (10) times two (2) to the power of N,where N is an integer.

In embodiments, the first amount of data streams and the second amountof data streams are equal to thirty-three (33) times two (2) to thepower of N, where N is an integer.

In embodiments, at least a portion of the first set of operations isperformed prior to step (k).

In embodiments, at least a portion of the first set of operations isperformed after step (k).

In embodiments, all of the first set of operations is performed afterstep (k).

In embodiments, the transmission of the second serial data stream fromthe serializer off the field programmable gate array is transmitted toan input/output module off the field programmable gate array.

In embodiments, the first serial data stream includes market data andthe second serial data stream includes trading data.

In embodiments, the step of generating the first wire rate clock signalfurther includes: (a) receiving, from the adjustable oscillator at atransceiver phase lock loop via a second reference clock pin in thefirst interface, the second clock signal; (b) processing, by thetransceiver phase lock loop, the received second clock signal; (c)providing, by the transceiver phase lock loop, the first wire rate clocksignal based on the processing of the received clock signal; and (d)transmitting, by the transceiver phase lock loop, the first wire rateclock signal to the serializer.

In embodiments, a phase lock loop generates the first wire rate clocksignal.

In embodiments, the phase lock loop is on the field programmable gatearray.

In embodiments, the phase lock loop that is part of the fieldprogrammable gate array system and not part of the field programmablegate array.

In embodiments, the step of generating the first wire rate clock signalfurther includes: (a) processing the second clock signal; (b) providingthe first wire rate clock signal based on the processing of the secondclock signal; and (c) transmitting, to the serializer, the first wirerate clock signal.

In embodiments, the sixth frequency corresponds to the third frequency.

In embodiments, the sixth phase is not aligned with the third phase.

The present invention also relates to a field programmable gate arraysystem comprising: (a) a field programmable gate array comprising (1) afirst interface comprising (A) a first reference clock pin, wherein saidfirst reference clock pin is configured to receive a first clock signalhaving a first frequency and a first phase; (B) a second reference clockpin, wherein said second reference clock pin is configured to receive asecond clock signal having a second frequency and a second phase; (C) afirst plurality of data pins, wherein said first plurality of data pinsis configured to receive a first serial data stream; (D) a secondplurality of data pins, wherein said second plurality of data pins isconfigured to transmit a second serial data stream; (2) a deserializeroperationally connected to (A) the first reference clock pin to receiveas a first input the first clock signal and (B) the first plurality ofdata pins to receive as a second input the first serial data stream, andwherein the deserializer is configured to (A) convert the first serialdata stream into a first plurality of parallel data streams having afirst amount of data streams; (B) generate a first receiver side clocksignal based on the first clock signal, wherein the first receiver sideclock signal has a third frequency and a third phase; and (C) transmitthe first plurality of parallel data streams and the first receiver sideclock signal within the field programmable gate array; (3) computationalcircuitry operationally connected to the deserializer to receive thefirst plurality of parallel data streams and the first receiver sideclock signal, wherein the computational circuitry is configured toperform a first set of operations on the first plurality of paralleldata streams to generate a second plurality of parallel processed datastreams having a second amount of data streams; (4) a serializeroperationally connected to (A) a transceiver phase lock loop to receiveas a third input a first wire rate clock signal based on the secondclock signal, wherein the first wire rate clock signal has a fourthfrequency and a fourth phase; (B) the second plurality of data pins totransmit as a first output the second serial data stream; (C) thecomputational circuitry, wherein the serializer receives the secondplurality of parallel processed data streams from the computationalcircuitry and the serializer transmits to the computational circuitry afirst transmitter side clock signal including a fifth frequency and afifth phase; and wherein the serializer is configured to (A) convert thesecond plurality of parallel processed data streams into the secondserial data stream; (B) generate the first transmitter side clock signalbased on the first wire rate signal, wherein the first transmitter sideclock signal has the fifth frequency and the fifth phase, wherein thefifth frequency is different than and less than the fourth frequency;and (C) transmit the second serial data stream to the second pluralityof data pins for transmission off the field programmable gate array; and(b) a phase control circuit, provided at least partially on the fieldprogrammable gate array, wherein the phase control circuit comprises (1)a phase detector configured to compare the third phase of the firstreceiver side clock signal to the fifth phase of the transmitter sideclock signal and to generate a phase difference indicator signal basedon a difference between the third phase of the first receiver side clocksignal and the fifth phase of the transmitter side clock signal; (2) aphase controller operationally connected to the phase detector andconfigured to receive the phase difference indicator signal, and whereinthe phase controller is configured to determine adjustment informationbased on the phase difference indicator signal; and (3) a phase lockloop with phase adjustment operationally connected to the phasecontroller and configured to receive the adjustment information as wellas operationally connected to the second reference clock pin of thefirst interface of the field programmable gate array, wherein the phaselock loop with phase adjustment is configured to generate the secondclock signal including the second frequency and the second phase basedon the adjustment information and transmit the second clock signal tothe second reference clock pin of the first interface of the fieldprogrammable gate array, wherein the transmitter side clock signal andthe first receiver side clock signal are phase aligned so that there isa fixed phase difference between the third phase and the fifth phase.

In embodiments, the third frequency corresponds to the first frequency.

In embodiments, the third phase is not aligned with the first phase.

In embodiments, the third frequency is different than the firstfrequency.

In embodiments, the first set of operations does not include clockdomain crossing operations that delays processing of the first set ofparallel data streams.

In embodiments, the first set of operations includes at least one of thefollowing: (i) an arithmetic operation; (ii) a logical operation; (iii)a pipeline operation; and (iv) a memory access operation.

In embodiments, the first amount of data streams is the same as thesecond amount of data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to one of the following: (i) eight (8) datastreams; (ii) ten (10) data streams; (iii) sixteen (16) data streams;(iv) twenty (20) data streams; (v) thirty-two (32) data streams; (vi)forty (40) data streams; (vii) sixty-four (64) data streams; (viii)eighty (80) data streams; (ix) one hundred twenty-eight (128) datastreams; and (x) one hundred sixty (160) data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to two (2) to the power of N, where N is aninteger.

In embodiments, the first amount of data streams and the second amountof data streams are equal to ten (10) times two (2) to the power of N,where N is an integer.

In embodiments, the first amount of data streams and the second amountof data streams are equal to thirty-three (33) times two (2) to thepower of N, where N is an integer.

In embodiments, the transceiver phase lock loop is operationallyconnected to the second reference clock pin and is configured to (A)receive the second clock signal, as a fourth input, to generate thefirst wire rate clock signal; and (B) transmit the first wire rate clocksignal as the third input, to the serializer.

In embodiments, the fourth frequency is equal to x times the fifthfrequency, where x is the second amount of data streams.

In embodiments, the fourth frequency is equal to x times the fifthfrequency, where x is half of the second amount of data streams.

In embodiments, the fourth frequency is a rational multiple of thesecond frequency.

In embodiments, the phase difference indicator signal is a pulse signal.

In embodiments, adjustment information indicates at least one of thefollowing:

-   -   (A) a desired phase; (B) a desired frequency; (C) a change in        phase; and (D) a change in frequency.

In embodiments, adjustment information comprises a voltage.

In embodiments, adjustment information comprise a digital transmission.

In embodiments, the adjustment information is used to do at least one ofthe following: (i) set a bias to an oscillator; (ii) set a dividerratio; and (iii) set a delay.

In embodiments, the first serial data stream comprises market data, thesecond serial data stream comprises order entry data, and the firstoperation comprises a trading algorithm.

In embodiments, the first serial data stream includes market data andthe second serial data stream includes trading data.

In embodiments, the field programmable gate array system furthercomprises: (5) a second interface comprising (A) a first clock outputpin configured to transmit the first receiver side clock signal, whereinthe first clock output pin is operationally connected to thedeserializer; and (B) a second clock output pin configured to transmitthe first transmitter side clock signal, wherein the second clock outputpin is operationally connected to the serializer; and wherein the phasedetector is not on the field programmable gate array and isoperationally connected to the first clock output pin and the secondclock output pin of the second interface of the field programmable gatearray.

In embodiments, the field programmable gate array system furthercomprises (a) a fourth plurality of data pins of the first interface,wherein the fourth plurality of data pins is configured to transmit afourth serial data stream; (b) a second deserializer operationallyconnected to: (i) the first reference clock pin to receive the firstclock signal as a third input to the second deserializer, wherein thesecond deserializer is configured to (A) receive a third serial datastream; (B) convert the third serial data stream into a third pluralityof parallel data streams having a third amount of data streams; (C)generate a second receiver side clock signal based on the first clocksignal, wherein the second receiver side clock signal has a sixthfrequency and a sixth phase; and (D) transmit within the fieldprogrammable gate array to the computational circuitry the thirdplurality of parallel data streams and the second receiver side clocksignal; (ii) the computational circuitry further configured to (A)receive the third plurality of parallel data streams and the secondreceiver side clock signal; and (B) generate a fourth plurality ofparallel processed data streams having a fourth amount of data streams;(c) a second serializer operationally connected to: (i) the transceiverphase lock loop to receive as a fifth input the first wire rate clocksignal based on the second clock signal; and (ii) the fourth pluralityof data pins to transmit as a second output the fourth serial datastream, wherein the second serializer is configured to (A) receive thefourth plurality of parallel processed data streams from thecomputational circuitry; (B) convert the fourth plurality of parallelprocessed data streams into the fourth serial data stream; (C) transmitthe fourth serial data stream to the fourth plurality of data pins fortransmission off the field programmable gate array; and (d) a secondphase control circuit, provided at least partially on the fieldprogrammable gate array, wherein the phase control circuit comprises (i)a second phase detector configured to (A) compare the third phase of thefirst receiver side clock signal to the sixth phase of the secondreceiver side clock signal; and (B) generate a second phase differenceindicator signal based on a difference between the third phase of thefirst receiver side clock signal and the sixth phase of the secondreceiver side clock signal; (ii) a second phase controller operationallyconnected to the second phase detector and configured to receive thesecond phase difference indicator signal, wherein the second phasecontroller is configured to determine second adjustment informationbased on the second phase difference indicator signal; and (iii) a delayelement operationally connected to the second phase controller and tothe second deserializer, wherein the second phase controller sends tothe delay element the second adjustment information, wherein the delayelement is configured to (A) delay the third serial data stream based onthe second adjustment information, and (B) transmit the third serialdata stream to the second deserializer, wherein the first receiver sideclock signal and the second receiver side clock signal are phase alignedso that there is a fixed phase difference between the third phase andthe sixth phase.

In embodiments, the field programmable gate array system furthercomprises a third plurality of data pins of the first interface, whereinthe delay element is placed on the field programmable gate array andoperationally connected to the third plurality of data pins and whereinthe delay element is further configured to (A) receive the third serialdata stream via the third plurality of data pins, and (B) transmit thethird serial data stream to the second deserializer.

In embodiments, the field programmable gate array system furthercomprises a third plurality of data pins of the first interface, wherein(A) the delay element is placed outside of the field programmable gatearray, (B) the delay element and the second deserializer areoperationally connected to the third plurality of data pins, and (C) thethird plurality of data pins is further configured to transmit the thirdserial data stream from the delay element to the second deserializer.

In embodiments, the field programmable gate array system furthercomprises (A) a first clock output pin of a second interface of thefield programmable gate array, the first clock output pin beingconfigured to transmit the first receiver side clock signal from thedeserializer to the second phase detector; and (B) a third clock outputpin of the second interface configured to transmit the second receiverside clock signal from the second deserializer to the second phasedetector, wherein the second phase detector is not placed on the fieldprogrammable gate array and is operationally connected to the firstclock output pin and the third clock output pin of the second interfaceof the field programmable gate array.

The present invention also relates to a method for processing a firstserial data stream comprising market data, using a field programmablegate array system, to generate a second serial data stream comprisingorder entry data, wherein the method comprises the steps of: (a)receiving, by a first plurality of data pins of a first interface of afield programmable gate array in the field programmable gate arraysystem, the first serial data stream; (b) receiving, by a firstreference clock pin in the first interface, a first clock signal havinga first frequency and a first phase; (c) transmitting, from the firstplurality of data pins of the first interface to a deserializer in thefield programmable gate array, the first serial data stream; (d)transmitting, from the first reference clock pin in the first interfaceto the deserializer, the first clock signal; (e) generating, by thedeserializer, a first receiver side clock signal having a secondfrequency and a second phase, based on the first clock signal; (f)converting, by the deserializer, the first serial data stream into afirst plurality of parallel data streams; (g) transmitting, from thedeserializer to computational circuitry in the field programmable gatearray, the first receiver side clock signal; (h) transmitting, from thedeserializer to the computational circuitry, the first plurality of datastreams comprising a first plurality of data items and a first amount ofdata streams; (i) transmitting, from the deserializer to a phasedetector of the field programmable gate array system, the first receiverside clock signal; (j) generating, using the field programmable gatearray system, a first transmitter side clock signal having a thirdfrequency and a third phase, by performing the following steps until afirst output of the phase detector is below a first threshold level: (i)generating, by a phase lock loop with phase adjustment in the fieldprogrammable gate array system, a second clock signal having a fourthfrequency and a fourth phase; (ii) generating a first wire rate clocksignal having a fifth frequency and a fifth phase based on the secondclock signal; (iii) generating, by a serializer in the fieldprogrammable gate array, an interim transmitter side clock signal havinga sixth frequency and a sixth phase; (iv) transmitting, from theserializer to the phase detector, the interim transmitter side clocksignal; (v) generating, by the phase detector, the first output based ona comparison of the first receiver side clock signal and the interimtransmitter side clock signal; (vi) transmitting, from the phasedetector to a phase controller of the field programmable gate arraysystem which is placed within the field programmable gate array, thefirst output; (vii) determining, by the phase controller, interimadjustment information based on the first output; and (viii)transmitting, from the phase controller to the phase lock loop withphase adjustment, the interim adjustment information; wherein, the phaselock loop with phase adjustment adjusts the second clock signal based onthe interim adjustment information and steps (i) through (viii) arerepeated until the first output of the phase detector is below the firstthreshold level; (k) transmitting, from the serializer to thecomputational circuitry, first transmitter side clock signal; (1)performing, by the computational circuitry, a first set of operations onat least a portion of the first plurality of data items to generate asecond plurality of parallel data streams comprising a second amount ofdata streams; (m) transmitting, from the computational circuitry to theserializer, the second plurality of parallel data streams; (n)converting, by the serializer, the second plurality of parallel datastreams into the second serial data stream; and (o) transmitting, fromthe serializer off the field gate programmable array system via a secondplurality of data pins of the first interface, the second serial datastream.

In embodiments, step (a) is performed before step (b).

In embodiments, step (b) is performed before step (a).

In embodiments, step (a) and step (b) are performed contemporaneously.

In embodiments, step (c) is performed before step (d).

In embodiments, step (d) is performed before step (c).

In embodiments, step (c) and step (d) are performed contemporaneously.

In embodiments, step (e) is performed before step (f).

In embodiments, step (f) is performed before step (e).

In embodiments, step (e) and step (f) are performed contemporaneously.

In embodiments, the second frequency corresponds to the first frequency.

In embodiments, the second phase is not aligned with the first phase.

In embodiments, the second frequency is different from the firstfrequency.

In embodiments, step (g) is performed before step (h).

In embodiments, step (h) is performed before step (g).

In embodiments, step (g) and step (h) are performed contemporaneously.

In embodiments, the phase detector is not placed on the fieldprogrammable gate array and the step (i) comprises transmitting, fromthe deserializer to the phase detector via a first clock output pin on asecond interface of the field programmable gate array, the firstreceiver side clock signal.

In embodiments, the step (i) further comprises transmitting, from thedeserializer to the phase detector, the first receiver side clock signalvia a first zero delay buffer phase lock loop of the field programmablegate array before transmitting the first receiver side clock signal viathe first clock output pin of the second interface.

In embodiments, the third frequency corresponds to the second frequency.

In embodiments, the third phase is aligned with the second phase.

In embodiments, a difference between the third phase and the secondphase is less than a second threshold level

In embodiments, the sixth frequency corresponds to the third frequency.

In embodiments, the sixth phase is not aligned with the third phaseprior to the first output of the phase detector being below the firstthreshold level.

In embodiments, the phase detector is not placed on the fieldprogrammable gate array and the step (j)(iv) comprises transmitting,from the serializer to the phase detector via a second clock output pinof the second interface of the field programmable gate array, theinterim transmitter side clock signal.

In embodiments, the step (j)(iv) further comprises transmitting, fromthe serializer to the phase detector, the interim transmitter side clocksignal via a second zero delay buffer phase lock loop of the fieldprogrammable gate array before transmitting the interim transmitter sideclock signal via the second clock output pin of the second interface.

In embodiments, the first set of operations does not include clockdomain crossing operations that delays processing of the first set ofparallel data streams.

In embodiments, the first amount of data streams is the same as thesecond amount of data streams.

In embodiments, the first amount of data streams and the second amountof data streams is one of the following: (i) eight (8) data streams;(ii) ten (10) data streams; (iii) sixteen (16) data streams; (iv) twenty(20) data streams; (v) thirty-two (32) data streams; (vi) forty (40)data streams; (vii) sixty-four (64) data streams; (viii) eighty (80)data streams; (ix) one hundred twenty-eight (128) data streams; and (x)one hundred sixty (160) data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to two (2) to the power of N, where N is aninteger.

In embodiments, the first amount of data streams and the second amountof data streams are equal to ten (10) times two (2) to the power of N,where N is an integer.

In embodiments, the first amount of data streams and the second amountof data streams are equal to thirty-three (33) times two (2) to thepower of N, where N is an integer.

In embodiments, at least a portion of the first set of operations isperformed prior to step (k).

In embodiments, at least a portion of the first set of operations isperformed after step (k).

In embodiments, all of the first set of operations is performed afterstep (k).

In embodiments, the transmission of the second serial data stream fromthe serializer off the field programmable gate array is transmitted toan input/output module off the field programmable gate array.

In embodiments, the first serial data stream includes market data andthe second serial data stream includes trading data.

In embodiments, the step of generating the first wire rate clock signalfurther includes: (a) receiving, from the phase lock loop with phaseadjustment at a transceiver phase lock loop via a second reference clockpin in the first interface, the second clock signal; (b) processing, bythe transceiver phase lock loop, the received second clock signal; (c)providing, by the transceiver phase lock loop, the first wire rate clocksignal based on the processing of the received clock signal; and (d)transmitting, by the transceiver phase lock loop, the first wire rateclock signal to the serializer.

In embodiments, a phase lock loop generates the first wire rate clocksignal.

In embodiments, the phase lock loop is on the field programmable gatearray.

In embodiments, the phase lock loop that is part of the fieldprogrammable gate array system and not part of the field programmablegate array.

In embodiments, the step of generating the first wire rate clock signalfurther includes: (a) processing the second clock signal; (b) providingthe first wire rate clock signal based on the processing of the secondclock signal; and (c) transmitting, to the serializer, the first wirerate clock signal.

In embodiments, the sixth frequency corresponds to the third frequency.

In embodiments, the sixth phase is not aligned with the third phase.

The present invention also relates to a field programmable gate arraysystem comprising (a) a field programmable gate array comprising (1) afirst interface comprising (A) a first reference clock pin, wherein saidfirst reference clock pin is configured to receive a first clock signalhaving a first frequency and a first phase; (B) a second reference clockpin, wherein said second reference clock pin is configured to receivethe first clock signal; (C) a first plurality of data pins, wherein saidfirst plurality of data pins is configured to receive a first serialdata stream; (D) a second plurality of data pins, wherein said secondplurality of data pins is configured to transmit a second serial datastream; (2) a deserializer operationally connected to (A) the firstreference clock pin to receive as a first input the first clock signaland (B) the first plurality of data pins to receive as a second inputthe first serial data stream, and wherein the deserializer is configuredto (A) convert the first serial data stream into a first plurality ofparallel data streams having a first amount of data streams; (B)generate a first receiver side clock signal based on the first clocksignal, wherein the first receiver side clock signal has a thirdfrequency and a third phase; and (C) transmit the first plurality ofparallel data streams and the first receiver side clock signal withinthe field programmable gate array; (3) computational circuitryoperationally connected to the deserializer to receive the firstplurality of parallel data streams and the first receiver side clocksignal, wherein the computational circuitry is configured to perform afirst set of operations on the first plurality of parallel data streamsto generate a second plurality of parallel processed data streams havinga second amount of data streams; (4) a serializer operationallyconnected to (A) an adjustable transceiver phase lock loop to receive asa third input a first wire rate clock signal, wherein the first wirerate clock signal has a fourth frequency and a fourth phase; (B) thesecond plurality of data pins to transmit as a first output the secondserial data stream; (C) the computational circuitry, wherein theserializer receives the second plurality of parallel processed datastreams from the computational circuitry and the serializer transmits tothe computational circuitry a first transmitter side clock signalincluding a fifth frequency and a fifth phase; and wherein theserializer is configured to (A) convert the second plurality of parallelprocessed data streams into the second serial data stream; (B) generatethe first transmitter side clock signal based on the first wire ratesignal, wherein the first transmitter side clock signal has the fifthfrequency and the fifth phase, wherein the fifth frequency is differentthan and less than the fourth frequency; and (C) transmit the secondserial data stream to the second plurality of data pins for transmissionoff the field programmable gate array; and (b) a phase control circuit,provided at least partially on the field programmable gate array,wherein the phase control circuit comprises (1) a phase detectorconfigured to compare the third phase of the first receiver side clocksignal to the fifth phase of the transmitter side clock signal and togenerate a phase difference indicator signal based on a differencebetween the third phase of the first receiver side clock signal and thefifth phase of the transmitter side clock signal; (2) a phase controlleroperationally connected to the phase detector and configured to receivethe phase difference indicator signal, and wherein the phase controlleris configured to determine adjustment information based on the phasedifference indicator signal; and (3) the adjustable transceiver phaselock loop operationally connected to the phase controller and configuredto receive the adjustment information as well as operationally connectedto the second reference clock pin of the first interface of the fieldprogrammable gate array, wherein the adjustable transceiver phase lockloop is configured to generate a second clock signal including thesecond frequency and the second phase based on the first clock signalreceived via the second reference clock pin and process the second clocksignal based on the adjustment information received from the phasecontroller to generate the first wire rate clock signal, wherein thetransmitter side clock signal and the first receiver side clock signalare phase aligned so that there is a fixed phase difference between thethird phase and the fifth phase.

In embodiments, the third frequency corresponds to the first frequency.

In embodiments, the third phase is not aligned with the first phase.

In embodiments, the third frequency is different than the firstfrequency.

In embodiments, the first set of operations does not include clockdomain crossing operations that delays processing of the first set ofparallel data streams.

In embodiments, the first set of operations includes at least one of thefollowing: (i) an arithmetic operation; (ii) a logical operation; (iii)a pipeline operation; and (iv) a memory access operation.

In embodiments, the first amount of data streams is the same as thesecond amount of data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to one of the following: (i) eight (8) datastreams; (ii) ten (10) data streams; (iii) sixteen (16) data streams;(iv) twenty (20) data streams; (v) thirty-two (32) data streams; (vi)forty (40) data streams; (vii) sixty-four (64) data streams; (viii)eighty (80) data streams; (ix) one hundred twenty-eight (128) datastreams; and (x) one hundred sixty (160) data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to two (2) to the power of N, where N is aninteger.

In embodiments, the first amount of data streams and the second amountof data streams are equal to ten (10) times two (2) to the power of N,where N is an integer.

In embodiments, the first amount of data streams and the second amountof data streams are equal to thirty-three (33) times two (2) to thepower of N, where N is an integer.

In embodiments, the fourth frequency is equal to x times the fifthfrequency, where x is the second amount of data streams.

In embodiments, the fourth frequency is equal to x times the fifthfrequency, where x is half of the second amount of data streams.

In embodiments, the fourth frequency is a rational multiple of thesecond frequency.

In embodiments, the phase difference indicator signal is a pulse signal.

In embodiments, adjustment information indicates at least one of thefollowing: (A) a desired phase; (B) a desired frequency; (C) a change inphase; and (D) a change in frequency.

In embodiments, adjustment information comprises a voltage.

In embodiments, adjustment information comprises a digital transmission.

In embodiments, the adjustment information is used to do at least one ofthe following: (i) set a bias to an oscillator; (ii) set a dividerratio; and (iii) set a delay.

In embodiments, the first serial data stream comprises market data, thesecond serial data stream comprises order entry data, and the firstoperation comprises a trading algorithm.

In embodiments, the first serial data stream includes market data andthe second serial data stream includes trading data.

In embodiments, the field programmable gate array system furthercomprises (5) a second interface comprising (A) a first clock output pinconfigured to transmit the first receiver side clock signal, wherein thefirst clock output pin is operationally connected to the deserializer;and (B) a second clock output pin configured to transmit the firsttransmitter side clock signal, wherein the second clock output pin isoperationally connected to the serializer, wherein the phase detector isnot on the field programmable gate array and is operationally connectedto the first clock output pin and the second clock output pin of thesecond interface of the field programmable gate array.

In embodiments, the field programmable gate array system furthercomprises (a) a fourth plurality of data pins of the first interface,wherein the fourth plurality of data pins is configured to transmit afourth serial data stream; (b) a second deserializer operationallyconnected to: (i) the first reference clock pin to receive the firstclock signal as a third input to the second deserializer, wherein thesecond deserializer is configured to (A) receive a third serial datastream; (B) convert the third serial data stream into a third pluralityof parallel data streams having a third amount of data streams; (C)generate a second receiver side clock signal based on the first clocksignal, wherein the second receiver side clock signal has a sixthfrequency and a sixth phase; and (D) transmit within the fieldprogrammable gate array to the computational circuitry the thirdplurality of parallel data streams and the second receiver side clocksignal; (ii) the computational circuitry further configured to (A)receive the third plurality of parallel data streams and the secondreceiver side clock signal; and (B) generate a fourth plurality ofparallel processed data streams having a fourth amount of data streams;(c) a second serializer operationally connected to: (i) the adjustabletransceiver phase lock loop to receive as a fifth input the first wirerate clock signal based on the second clock signal; and (ii) the fourthplurality of data pins to transmit as a second output the fourth serialdata stream, wherein the second serializer is configured to (A) receivethe fourth plurality of parallel processed data streams from thecomputational circuitry; (B) convert the fourth plurality of parallelprocessed data streams into the fourth serial data stream; (C) transmitthe fourth serial data stream to the fourth plurality of data pins fortransmission off the field programmable gate array; and (d) a secondphase control circuit, provided at least partially on the fieldprogrammable gate array, wherein the phase control circuit comprises (i)a second phase detector configured to (A) compare the third phase of thefirst receiver side clock signal to the sixth phase of the secondreceiver side clock signal; and (B) generate a second phase differenceindicator signal based on a difference between the third phase of thefirst receiver side clock signal and the sixth phase of the secondreceiver side clock signal; (ii) a second phase controller operationallyconnected to the second phase detector and configured to receive thesecond phase difference indicator signal, wherein the second phasecontroller is configured to determine second adjustment informationbased on the second phase difference indicator signal; and (iii) a delayelement operationally connected to the second phase controller and tothe second deserializer, wherein the second phase controller sends tothe delay element the second adjustment information, wherein the delayelement is configured to (A) delay the third serial data stream based onthe second adjustment information, and (B) transmit the third serialdata stream to the second deserializer, wherein the first receiver sideclock signal and the second receiver side clock signal are phase alignedso that there is a fixed phase difference between the third phase andthe sixth phase.

In embodiments, the field programmable gate array system furthercomprises a third plurality of data pins of the first interface, whereinthe delay element is placed on the field programmable gate array andoperationally connected to the third plurality of data pins and whereinthe delay element is further configured to (A) receive the third serialdata stream via the third plurality of data pins, and (B) transmit thethird serial data stream to the second deserializer.

In embodiments, the field programmable gate array system furthercomprises a third plurality of data pins of the first interface, wherein(A) the delay element is placed outside of the field programmable gatearray, (B) the delay element and the second deserializer areoperationally connected to the third plurality of data pins, and (C) thethird plurality of data pins is further configured to transmit the thirdserial data stream from the delay element to the second deserializer.

In embodiments, the field programmable gate array system furthercomprises (A) a first clock output pin of a second interface of thefield programmable gate array, the first clock output pin beingconfigured to transmit the first receiver side clock signal from thedeserializer to the second phase detector; and (B) a third clock outputpin of the second interface configured to transmit the second receiverside clock signal from the second deserializer to the second phasedetector, wherein the second phase detector is not placed on the fieldprogrammable gate array and is operationally connected to the firstclock output pin and the third clock output pin of the second interfaceof the field programmable gate array.

The present invention also relates to a method for processing a firstserial data stream comprising market data, using a field programmablegate array system, to generate a second serial data stream comprisingorder entry data, wherein the method comprises the steps of (a)receiving, by a first plurality of data pins of a first interface of afield programmable gate array in the field programmable gate arraysystem, the first serial data stream; (b) receiving, by a firstreference clock pin in the first interface, a first clock signal havinga first frequency and a first phase; (c) transmitting, from the firstplurality of data pins of the first interface to a deserializer in thefield programmable gate array, the first serial data stream; (d)transmitting, from the first reference clock pin in the first interfaceto the deserializer, the first clock signal; (e) generating, by thedeserializer, a first receiver side clock signal having a secondfrequency and a second phase, based on the first clock signal; (f)converting, by the deserializer, the first serial data stream into afirst plurality of parallel data streams; (g) transmitting, from thedeserializer to computational circuitry in the field programmable gatearray, the first receiver side clock signal; (h) transmitting, from thedeserializer to the computational circuitry, the first plurality of datastreams comprising a first plurality of data items and a first amount ofdata streams; (i) transmitting, from the deserializer to a phasedetector of the field programmable gate array system, the first receiverside clock signal; (j) generating, using the field programmable gatearray system, a first transmitter side clock signal having a thirdfrequency and a third phase, by performing the following steps until afirst output of the phase detector is below a first threshold level: (i)receiving, by an adjustable transceiver phase lock loop via a secondreference clock pin in the first interface, the first clock signal togenerate a second clock signal having a fourth frequency and a fourthphase; (ii) generating a first wire rate clock signal having a fifthfrequency and a fifth phase based on the second clock signal and interimadjustment information; (iii) generating, by a serializer in the fieldprogrammable gate array, an interim transmitter side clock signal havinga sixth frequency and a sixth phase; (iv) transmitting, from theserializer to the phase detector, the interim transmitter side clocksignal; (v) generating, by the phase detector, the first output based ona comparison of the first receiver side clock signal and the interimtransmitter side clock signal; (vi) transmitting, from the phasedetector to a phase controller of the field programmable gate arraysystem which is placed within the field programmable gate array, thefirst output; (vii) determining, by the phase controller, the interimadjustment information based on the first output; and (viii)transmitting, from the phase controller to the adjustable transceiverphase lock loop, the interim adjustment information, wherein, theadjustable transceiver phase lock loop adjusts the second clock signalbased on the interim adjustment information and steps (i) through (viii)are repeated until the first output of the phase detector is below thefirst threshold level; (k) transmitting, from the serializer to thecomputational circuitry, first transmitter side clock signal; (1)performing, by the computational circuitry, a first set of operations onat least a portion of the first plurality of data items to generate asecond plurality of parallel data streams comprising a second amount ofdata streams; (m) transmitting, from the computational circuitry to theserializer, the second plurality of parallel data streams; (n)converting, by the serializer, the second plurality of parallel datastreams into the second serial data stream; and (o) transmitting, fromthe serializer off the field gate programmable array system via a secondplurality of data pins of the first interface, the second serial datastream.

In embodiments, step (a) is performed before step (b).

In embodiments, step (b) is performed before step (a).

In embodiments, step (a) and step (b) are performed contemporaneously.

In embodiments, step (c) is performed before step (d).

In embodiments, step (d) is performed before step (c).

In embodiments, step (c) and step (d) are performed contemporaneously.

In embodiments, step (e) is performed before step (f).

In embodiments, step (f) is performed before step (e).

In embodiments, step (e) and step (f) are performed contemporaneously.

In embodiments, the second frequency corresponds to the first frequency.

In embodiments, the second phase is not aligned with the first phase.

In embodiments, the second frequency is different from the firstfrequency.

In embodiments, step (g) is performed before step (h).

In embodiments, step (h) is performed before step (g).

In embodiments, step (g) and step (h) are performed contemporaneously.

In embodiments, the phase detector is not placed on the fieldprogrammable gate array and the step (i) comprises transmitting, fromthe deserializer to the phase detector via a first clock pin on a secondinterface of the field programmable gate array, the first receiver sideclock signal.

In embodiments, the step (i) further comprises transmitting, from thedeserializer to the phase detector, the first receiver side clock signalvia a first zero delay buffer phase lock loop of the field programmablegate array before transmitting the first receiver side clock signal viathe first clock output pin of the second interface.

In embodiments, the third frequency corresponds to the second frequency.

In embodiments, the third phase is aligned with the second phase.

In embodiments, a difference between the third phase and the secondphase is less than a second threshold level

In embodiments, the sixth frequency corresponds to the third frequency.

In embodiments, the sixth phase is not aligned with the third phaseprior to the first output of the phase detector being below the firstthreshold level.

In embodiments, the phase detector is not placed on the fieldprogrammable gate array and the step (j)(iv) comprises transmitting,from the serializer to the phase detector via a second clock output pinof the second interface of the field programmable gate array, theinterim transmitter side clock signal.

In embodiments, the step (j)(iv) further comprises transmitting, fromthe serializer to the phase detector, the interim transmitter side clocksignal via a second zero delay buffer phase lock loop of the fieldprogrammable gate array before transmitting the interim transmitter sideclock signal via the second clock output pin of the second interface.

In embodiments, the first set of operations does not include clockdomain crossing operations that delays processing of the first set ofparallel data streams.

In embodiments, the first amount of data streams is the same as thesecond amount of data streams.

In embodiments, the first amount of data streams and the second amountof data streams is one of the following: (i) eight (8) data streams;(ii) ten (10) data streams; (iii) sixteen (16) data streams; (iv) twenty(20) data streams; (v) thirty-two (32) data streams; (vi) forty (40)data streams; (vii) sixty-four (64) data streams; (viii) eighty (80)data streams; (ix) one hundred twenty-eight (128) data streams; and (x)one hundred sixty (160) data streams.

In embodiments, the first amount of data streams and the second amountof data streams are equal to two (2) to the power of N, where N is aninteger.

In embodiments, the first amount of data streams and the second amountof data streams are equal to ten (10) times two (2) to the power of N,where N is an integer.

In embodiments, the first amount of data streams and the second amountof data streams are equal to thirty-three (33) times two (2) to thepower of N, where N is an integer.

In embodiments, at least a portion of the first set of operations isperformed prior to step (k).

In embodiments, at least a portion of the first set of operations isperformed after step (k).

In embodiments, all of the first set of operations is performed afterstep (k).

In embodiments, the transmission of the second serial data stream fromthe serializer off the field programmable gate array is transmitted toan input/output module off the field programmable gate array.

In embodiments, the first serial data stream includes market data andthe second serial data stream includes trading data.

In embodiments, the step of generating the first wire rate clock signalfurther includes: (a) generating, by the adjustable phase lock loop, thesecond clock signal based on the received first clock signal; (b)processing, by the adjustable transceiver phase lock loop, the secondclock signal based on the interim adjustment information received fromthe phase controller; (c) providing, by the adjustable transceiver phaselock loop, the first wire rate clock signal based on the processing ofthe second clock signal; and (d) transmitting, by the adjustabletransceiver phase lock loop, the first wire rate clock signal to theserializer.

In embodiments, a phase lock loop generates the first wire rate clocksignal.

In embodiments, the phase lock loop is on the field programmable gatearray.

In embodiments, the phase lock loop that is part of the fieldprogrammable gate array system and not part of the field programmablegate array.

In embodiments, the step of generating the first wire rate clock signalfurther includes: (a) processing the second clock signal; (b) providingthe first wire rate clock signal based on the processing of the secondclock signal; and (c) transmitting, to the serializer, the first wirerate clock signal.

In embodiments, the sixth frequency corresponds to the third frequency.

In embodiments, the sixth phase is not aligned with the third phase.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described withreference to the accompanying figures, wherein:

FIG. 1 is a block diagram of a conventional field programmable gatearray;

FIG. 1A is a block diagram of an exemplary transceiver suitable for usein the field programmable gate array of FIG. 1;

FIG. 1B is an exemplary block diagram of a Clock Domain Crossing Circuitsuitable for use in the field programmable gate array of FIG. 1;

FIG. 1C illustrates exemplary input output waveforms of the Clock DomainCrossing Circuit of the field programmable gate array of FIG. 1;

FIG. 2A illustrates an example of a phase difference between thereceiver side clock and the transmitter side clock of an FPGA inaccordance with an exemplary embodiment of the present invention;

FIG. 2B illustrates an example of a phase difference between receiverclock and a transmitter clock of an FPGA in accordance with an exemplaryembodiment of the present invention;

FIG. 2C illustrates an example of a phase difference between receiverclock and a transmitter clock of an FPGA in accordance with an exemplaryembodiment of the present invention;

FIG. 2D illustrates an example of an inverted phase difference betweenreceiver clock and a transmitter clock of an FPGA in accordance with anexemplary embodiment of the present invention;

FIG. 3A is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 3B is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 3C is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 3D is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 3E is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 3F is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 4A is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 4B is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 4C is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 5A is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 5B is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 5C is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 5D is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 5E is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 5F is a block diagram of a field programmable gate array system inaccordance with an exemplary embodiment of the present invention;

FIG. 6A is an exemplary block diagram of a deserializer suitable for usein the field programmable gate array systems of FIGS. 3A-3F, 4A-4C and5A-5F in accordance with an exemplary embodiment of the presentinvention;

FIG. 6B is an exemplary circuit diagram of a continuous-time linearequalizer (CTLE) suitable for use in the deserializer of FIG. 6A inaccordance with an exemplary embodiment of the present invention;

FIG. 6C is an exemplary block diagram of a clock and data recovery (CDR)circuit suitable for use in the deserializer of FIG. 6A in accordancewith an exemplary embodiment of the present invention;

FIG. 6D is an exemplary circuit diagram of a decision circuit suitablefor use in the deserializer of FIG. 6A in accordance with an embodimentof the present invention;

FIG. 6E is an exemplary block diagram of a deserializer suitable for usein the field programmable gate array systems of FIGS. 3A-3F, 4A-4C and5A-5F in accordance with an exemplary embodiment of the presentinvention;

FIG. 7A is an exemplary block diagram of a serializer suitable for usein the field programmable gate array systems of FIGS. 3A-3F, 4A-4C and5A-5F in accordance with an exemplary embodiment of the presentinvention;

FIG. 7B is an exemplary block diagram of a serializer suitable for usein field programmable gate array systems of FIGS. 3A-3F, 4A-4C and 5A-5Fin accordance with an exemplary embodiment of the present invention;

FIG. 8A is an exemplary block diagram of a phase detector suitable foruse in the field programmable gate array systems of FIGS. 3A-3F, 4A-4Cand 5A-5F in accordance with an exemplary embodiment of the presentinvention;

FIG. 8B is another exemplary block diagram of a phase detector suitablefor use in the field programmable gate array systems of FIGS. 3A-3F,4A-4C and 5A-5F in accordance with an exemplary embodiment of thepresent invention;

FIG. 8C is another exemplary block diagram of a phase detector suitablefor use in the field programmable gate array systems of FIGS. 3A-3F,4A-4C and 5A-5F in accordance with an exemplary embodiment of thepresent invention;

FIG. 8D is another exemplary block diagram of a phase detector suitablefor use in the field programmable gate array systems of FIGS. 3A-3F,4A-4C and 5A-5F in accordance with an exemplary embodiment of thepresent invention;

FIGS. 9A-C are flow charts of a process implementing the fieldprogrammable gate array system in accordance with an exemplaryembodiment of the present invention;

FIGS. 10A-10B are flow charts of a process for generating a transmitterside clock signal in accordance with an exemplary embodiment of thepresent invention;

FIG. 11 is a flow chart of a process for generating a wire rate clocksignal in accordance with an embodiment of the present invention;

FIGS. 12A-C are flow charts of a process implementing the fieldprogrammable gate array system in accordance with an exemplaryembodiment of the present invention;

FIGS. 13A-13B are flow charts of a process for generating a transmitterside clock signal in accordance with an exemplary embodiment of thepresent invention;

FIG. 14 is a flow chart of a process for generating a wire rate clocksignal in accordance with an embodiment of the present invention;

FIGS. 15A-C are flow charts of a process implementing the fieldprogrammable gate array system in accordance with an exemplaryembodiment of the present invention;

FIGS. 16A-16B are flow charts of a process for generating a transmitterside clock signal in accordance with an exemplary embodiment of thepresent invention; and

FIG. 17 is a flow chart of a process for generating a wire rate clocksignal in accordance with an exemplary embodiment of the presentinvention.

DETAILED DESCRIPTION

The present invention generally relates to a field programmable gatearray system. In embodiments, the present invention generally relates toa field programmable gate array and an external or internal phasecontroller providing phase matching between a receiver clock and atransmitter clock used in the field programmable gate array.

FIG. 1 illustrates an exemplary schematic of a FPGA 100. Examples ofcurrently available FPGAs include the XCVU3P-2FFVC1517E, Intel1SG280LH3F55E3VG, Xilinx XCVU9P-2FLGA2104E the Vitrex® Ultrascale, theVitrex® Ultrascale Plus, the Stratix® V, the Stratix® 10, XILINX XC3042,the Intel EP4CE6E22C8LN and the Lattice ICE40LP384-SG32, to name a few.

In embodiments, FPGA 100 may include an FPGA Core 106 and a number ofperipheral systems. In embodiments, the FPGA Core 106 includes 4 maincomponents: logic elements, digital signal processor blocks (“DSPblocks”), clock distribution components, and memories, to name a few,and may generally be referred to as the FPGA core, or simply logic core,106. In embodiments, the peripheral systems may include CPU cores,reconfiguration controllers, security features, hardened logicfunctions, FPGA transceiver banks 102, hardened high-speed interfaceaccelerators (e.g., PCIe or 100 GbE controllers, to name a few),general-purpose I/O pins, memory interface controllers, larger memories,analog components (e.g., ADCs or DACs to name a few), and 3D-stackedmemories (e.g., HBM), to name a few. The FPGA Transceiver banks 102receive serial data to be processed from outside the FPGA and thentransmit serial data out of the FBGA after being processed by internalcircuitry of the FPGA 100. Some FPGA's may not include FPGA Transceiverbanks.

In embodiments, data may be received by a receiver side of thetransceiver banks 102 in the FPGA 100 at rates such as 10 or 25 Gbps, toname a few. In embodiments, rates may be between a range of 9-25 Gbps,10-25 Gbps, 22-33 Gbps, 33-45 Gbps, 45-60 Gbps, 60-80 Gbps, or 80-120Gbps, to name a few. An exemplary transceiver (including deserializer104′ and serializer 110′) suitable for use in FPGA 100 is shown in FIG.1A. In embodiments, the input data may be received from I/O module 120.The input data may then be deserialized, using deserializer 104, forexample, and parallelized into parallel data streams, such as 16-64 bitswide at 1/16^(th) to 1/64^(th) of the frequency of the incoming datastream, to name a few. In embodiments, FPGA transceivers may supportparallel streams with widths of, for example, 8 bits, 10 bits, 16 bits,20 bits, 32 bits, 40 bits, 64 bits, 80 bits, 128 bits, and 160 bits(with corresponding clock division), to name a few. In some FPGAs, aphysical coding sublayer (PCS) may be used in the transceiver 102. Insome FPGAs, the PCS of the transceiver 102 may be bypassed and so thatthe corresponding PCS functions may be done in the FPGA core 106. Afterdeserialization, data is available for computation in the FPGA logiccore 106 where computation is completed. In embodiments, the computationof the data may be computed by receive-side computation 110 a of theFPGA core 106. Similarly, data may be computed by the transmit-sidecomputation 110 b.

In embodiments, a REFERENCE CLOCK signal is provided, by Oscillator orClock Generator 122, to both the deserializer 104 and the serializer110. The REFERENCE CLOCK signal is received by the serializer 110 viathe transceiver PLL 108 (Phase-Locked Loop). The first receiver sideclock signal RXCLOCK, however, is different in both frequency and phasefrom the REFERENCE CLOCK based on the deserialization process thatoccurs in the deserializer 104. Similarly, the transmitter clock signalTXCLOCK, while based on the REFERENCE CLOCK signal varies in frequencyand phase based on processing that occurs in the serializer 110. As aresult, the RXCLOCK signal and TXCLOCK signal will be out of phase. Aknown solution to this issue is the inclusion of the RX to TX clockdomain crossing circuit 112. An exemplary Clock Domain Crossing circuitis suitable for use in FPGA 100 is shown as clock domain crossingcircuit 112′ in connection with FIG. 1B. As is noted above, theinclusion of the RX to TX clock domain crossing circuit 112 creates atechnical problem, introducing an inherent delay in the FPGA 100, whichis undesirable. The technical problem of an inherent delay isillustrated in FIG. 1C. FIG. 1C illustrates an example of a phasedifference between the receiver side clock and the transmitter sideclock of the conventional field programmable gate array of FIG. 1.

Thereafter, the data it transmitted out of the FPGA 100. After allcomputation (computation by receive-side computation 110 a and/ortransmit-side computation 110 b) and clock domain crossing (by RX to TXclock domain crossing 112) is completed, the data goes back to thetransmit side of the FPGA transceiver 102 to be serialized by theserializer 110 out on an output wire. The serializer 110 (e.g., thetransmitter) typically runs off a fast clock generated (e.g., 10 Gbps,25 Gbps, or a range from 10 Gbps to 25 Gbps, or a range from 1 Gbps to100 Gbps, to name a few) by the transceiver phase-locked loop 108 withinthe transceiver bank 102 from a reference clock signal received byoscillator or clock generator 122.

The receiver (deserializer 104) and transmitter (serializer 110) of thetransceiver 102 operate on different clocks that are independentlygenerated within the transceiver 102. The receiver clock domain istypically generated by a clock and data recovery (CDR) circuit from theincoming data stream. The transmitter clock is typically generated bythe transceiver phase-locked loop 108. In conventional FPGAs, phasematching or synchronizing is provided using the clock domain crossingcircuit 112 that adjusts the phases of the two clock domains. The clockdomain crossing circuit 112 may be an asynchronous FIFO or anasynchronous gearbox, to name a few. In embodiments, domain crossingcircuit 112 may be a mesochronous clock crossing circuit. Inembodiments, clock domain crossing circuit 112 may be instantiatedwithin transceiver 102 of FPGA 100, although this structure has a higherlatency cost than implementing it in logic fabric 106 of FPGA 100. Asdiscussed above, a significant drawback of the clock domain crossingcircuit 112 is that it adds latency related to the phase differencebetween the clocks plus the latency of the synchronizers used, and doesnot perform any computation, such that it slows the effective processingspeed of FPGA 100.

FIGS. 2A-2D illustrate example phase differences between the receiverside clock and the transmitter side clock. It will be understood thatother examples of phase differences may be present consistent with theteachings of this disclosure. Generally, when the clocks are aligned,the minimum and maximum delays may be used to determine the timeavailable for the computation of data.

FIG. 2A illustrates an example of a phase difference between thereceiver side clock and the transmitter side clock of an FPGA inaccordance with an embodiment. FIG. 2A depicts a phase difference whenthe clocks are under 360 degrees apart. This embodiment results inminimum delay 2106A and maximum delay 2108A, for which data may becomputed. The minimum delay 2106A is from the driving clock edge back toitself, as shown in FIG. 2A.

FIG. 2B illustrates an example of a phase difference between receiverclock and a transmitter clock of an FPGA in accordance with anembodiment. The phase difference depicted in FIG. 2B shows a phasedifference with medium delay 2106B and maximum delay 2108B. The exampleshown in FIG. 2B shows a minimum delay 2106B of 0. However, in practice,while designers may design for the phase difference in FIG. 2B, thephase difference may end up more similar to the phase differences ofFIGS. 2A and/or 2C.

FIG. 2C illustrates an example of a phase difference between receiverclock and a transmitter clock of an FPGA in accordance with anembodiment. FIG. 2C depicts a phase difference with the clocks are over360 degrees apart. The time between minimum delay 2106C and maximumdelay 2108C may allow for more data computations within the phasedifference.

FIG. 2D illustrates an example of an inverted phase difference betweenreceiver clock and a transmitter clock of an FPGA in accordance with anembodiment. This example of an inverted phase difference between minimumdelay 2106D and maximum delay 2108D shows that, even inverted, the phasedifference would work similarly to the phase detectors of FIGS. 2A-2C.

FIG. 3A illustrates a block diagram of a field programmable gate array(FPGA) system in accordance with an embodiment of the present disclosurethat provides phase control while minimizing latency. In embodiments,the system generally includes FPGA 3100 which preferably includes FPGAtransceiver banks 3102. In embodiments, the FPGA transceiver banks 3102includes deserializer 3104 on the receiver end of FPGA 3100, transceiverPLL 3108, and serializer 3110 on the transmitter end of FPGA 3100.Deserializer 3104 may be similar to the deserializer 3104′ anddeserializer 3104″ discussed below in connection with FIGS. 6A-6E, thedescriptions of which applying herein. Serializer 3110 may be similar toserializer 3110′ and serializer 3110″ discussed below in connection withFIGS. 7A and 7B respectively, the descriptions of which applying herein.In embodiments, deserializer 3104 receives a serial data stream from theI/O module 3120 and provides deserialized data to the FPGA core 3106.More specifically, the deserializer 3104 will provide parallel datastreams to the FPGA core 3106 for processing. As illustrated, thisparallel data is provided to the computational logic 3210 whereprocessing and computation takes place. FIG. 3A illustratescomputational logic 3210 as a single element with a receiver side and atransmitting side. In embodiments, the computational logic 3210 may beimplemented as a plurality of logic elements. In embodiments, when allcomputation is complete, the resulting data may be provided toserializer 3110 and then transmitted out of FPGA 3100, preferably viaI/O module 3120, for example. In embodiments, the I/O module may be adirect soldered cable, on-chip optics, or an on-board opticaltransceiver, to name a few. Additionally, the I/O module may be an SFP,a QSFP, a micro QSFP, or a QSFP-DD, to name a few.

In embodiments, the serializer 3110 and deserializer 3104 (SERDES)circuit(s) may contain two major sections: (1) an analog side whosepurpose is signal cleaning, and (2) a digital side which turns theanalog signal into bits and converts between parallel and serial datastreams. In embodiments, the analog side may include a few differenttypes of amplifiers to provide signal cleaning.

In embodiments, as shown in FIG. 3A, a PLL with phase adjustment 3300may be located inside the FPGA 3100 fabric with adjustable feedbackdividers, which in turn adjust the frequency of the PLL. Alternatively,the PLL with phase adjustment 3300 may comprise a phase adjuster on theoutput of the loop, which adjusts the phase of the PLL directly. Inembodiments, the same description may apply to adjustable transceiverPLLs 3108 and 5108 respectively shown in FIGS. 3B and 5B, but thedifferences are the location of the PLL inside the FPGA (e.g., withinthe FPGA core 3106 versus outside the FPGA core and within the FPGAtransceiver banks 3102, 5102) and the operating frequency.

As illustrated in FIG. 3A, in embodiments, the first receiver side clocksignal RXCLOCK is based on but has a different frequency and phase thanthe REFERENCE CLOCK signal provided by oscillator 3000 based on theoperation of the deserializer 3104, as is discussed above with respectto the serializer 104. In embodiments, the transmitter side clock signalTXCLOCK provided from the serializer 3110 may be provided by transceiverphase locked loop 3108, which is provided inside FPGA 3100, and may becontrolled by Phase Lock Loop (PLL) with Phase Adjustment 3300 of theFPGA core 3106. In this manner, the frequency or phase of thetransmitter side clock signal TXCLOCK may be adjusted.

In embodiments, an internal phase controller 3202 is connected to thePLL with Phase Adjustment 3300 and provides control signals to allow foradjustment of the phase of at least the transmitter side clock signalTXCLOCK. The phase of the incoming data, in embodiments, may be adjustedby adjusting the data stream received by I/O module 3120. Inembodiments, the internal phase controller 3202 provides control signalsbased on the phase difference between the receiver clock signal RXCLOCKand the transmitter clock signal TXCLOCK. In embodiments, the phasedifference may be determined using phase detector 3206

Using the configuration of FIG. 3A, the receiver clock signal RXCLOCKand the transmitter clock signal TXCLOCK may be sufficiently aligned inphase such that there is no need for the clock domain crossing circuitdiscussed above, eliminating a technical problem. The technical solutionto a technical problem, in embodiments, is shown in the system of FIG.3A, which provides a wire or a logic path via the controller 3202 andPLL with phase adjustment 3300 to adjust the phase of the transmitterside clock TXCLOCK to correspond to that of the first receiver sideclock signal RXCLOCK. Some delays may be introduced in the pathincluding min_delay and max_delay constraints of the wire andcomponents, however, such delays are insignificant compared to thelarger delays that are necessarily present when an asynchronous clockdomain crossing circuit is used.

In embodiments, with a phase difference between about 45° and 360°, themin_delay should be set at 0 and the max_delay should be set equal tothe time between rising edges on the receive clock signal and transmitclock signal (as shown in FIGS. 2A and 2B). The difference between 45°and 360° approximates the minimum amount of time required to safelycapture data. In embodiments, the formula for the time is as follows:t _(min) =t _(setup) +t _(wire) +t _(clock-out) +Δt _(jitter) +t _(skew)

Referencing the above equation, t_(setup) is the setup time of aflip-flop (i.e., a latch) on FPGA 3100. The t_(wire) is the delay ofthe, for example, shortest wire (as compared to other wires on the FPGA)on FPGA 3100. In embodiments, the shortest wire is the shortest possiblewire. In embodiments, t_(clock-out) is the clock-to-output time of aflip-flop on the FPGA 3100. In embodiments, Δt_(jitter) is a safetyfactor for jitter on both clocks. Jitter, for example, is the deviationfrom true periodicity of a periodic signal from a reference clock. Inembodiments, t_(skew) is the skew between clock paths to the twoflip-flops on FPGA 3100.

In embodiments where there is a phase difference with the transmitterclock running less than 45° ahead of the receiver clock, metastabilitymay occur unless the clocks are treated as being greater than 360degrees apart (as seen in FIG. 2C). In such embodiments, the min and maxdelay are set with the min delay greater than the time differencebetween rising edges and the max delay less than one full clock cycleplus the min delay. In embodiments, the transmit clock signal may alsosample on the opposite edge as the receiving clock signal launch edge,but cases are symmetric. In embodiments, a change from positive tonegative edge sampling adds an approximate 180 degrees offset. Inembodiments, manufacturer analysis tools may add a reasonable jittermargin to account for times when jitter causes temporary misalignment.

In embodiments, to avoid instability, the transceiver PLL 3108preferably runs at the highest bandwidth available from the PLL on theFPGA, while the overall phase locking system including the phasedetectors 3206, controller 3202 and PLL with Phase Adjustment 3300 runat a low bandwidth. In embodiments, the highest bandwidth available fromthe PLL on the FPGA may range between 100 kHz and 10 MHz. In suchembodiments, the poles in the on-chip PLLs on the FPGA 3100 areprevented from affecting the gain or phase margin of the overall controlloop which may be included in the controller 3202. In embodiments, anyloop filter order may be used in the controller 3202 as long as thebandwidth is low enough to avoid interaction of poles in the controlsystem. In embodiments, a low bandwidth may be, for example, 100 Hz-40kHz, 400 Hz and 4 khz, 4 kHz and 10 kHz, 100 hz to 40 kHz, 0 Hz and 100Hz, to name a few. In embodiments, a second-order loop filter may beused in an effort to improve lock time and phase locking error. Inembodiments a third-order filter may be used if the second-order filteris stable. In embodiments, a first-order filter is also an option whenthe second-order filter has stability problems. In embodiments, otherorder filters may be used, depending on the internal bandwidth of theFPGA 3100.

In embodiments, where the bandwidth of the controller 3202 is too low,the receiving clock and the transmitter clock may not track each otherclosely enough to consider them locked. In embodiment, such trackingissues may be solved by adjusting the min and max delay constraintsappropriately to provide a cushion for the clocks being away from theexpected phase. In embodiments, a PD (Phase Detector) control system maybe provided after the loop filter to aid in resolving tracking issues.In embodiments, tracking issues may be addressed by adding somenonlinearity in the transfer function of the filter. Other solutions mayalso be possible.

In the embodiment of FIG. 3A, the controller 3202 utilizes aphase-locked loop. In embodiments, a derivative component may beprovided in the loop filter to slow down the control loop when theoutput value has changed recently and reduces the chance that the PLLson the FPGA will lose lock. In embodiments, the derivative component'scoefficient will be negative (i.e., the derivative component will actagainst the loop filter, however, some references refer to this as beinga positive direction) and less than 1 (so the path straight from thefilter dominates). The loop filter consists of a standard PLL loopfilter with bandwidth between 400 Hz and 40 kHz of up to 4th order plusa derivative component that slows down the loop based on recent changesin the filter value. The bandwidth of the standard PLL loop filter, insome embodiments, may range between 400 Hz and 4 kHz, 4 kHz and 10 kHz,100 Hz and 40 kHz, 0 Hz and 100 Hz, to name a few. The extra derivativecomponent helps to keep the other components in the control system intheir linear operating regions. The loop filter is represented byDigitalFilter( ) in the pseudocode provided below.

While similar to the embodiment shown in FIG. 3A, in FIG. 3B, aninternal phase controller 3202 is directly connected and provides phasecontrol signals to the adjustable transceiver PLL 3108, which is locatedin the FPGA transceiver banks 3102 and outside the FPGA core 3106. FIG.3B illustrates a block diagram of an FPGA system in accordance with anembodiment of the present invention that provides phase control whileminimizing latency wherein the FPGA 3100 has an acceptablephase-controlled adjustable transceiver PLL 3108. This embodiment usesthe phase-controlled transceiver PLL directly to control the phase ofTXCLOCK. An acceptable phase-control system must have adjustment gaincapable of matching phase transients on the incoming data stream towithin the timing margin (train), meaning that the following inequalitymust be able to be satisfied:

∫₀^(∞)Δ t_(p k)(ω)[1 − LF(ω)] d ω ≤ t_(min)

Where Δt_(pk)(ω) is the peak design jitter at that frequency and LF isthe transfer function over frequency of the phase locking systemconsisting of the phase detector 3206, controller 3202, and adjustabletransceiver PLL 3108. The transfer function of a phase locked looprelates linearly to the phase control gain at low frequencies, and isgiven by:

${{LF}(\omega)} = \frac{K_{APLL}K_{PD}{{T(\omega)}/{F(\omega)}}}{\omega^{2} + {K_{APLL}K_{P\; D}{{T(\omega)}/{F(\omega)}}}}$where K_(APLL) is the phase control gain of adjustable PLL 3108, K_(PD)is the gain of the phase detector 3206, T(ω) is the transfer function ofthe PLL controller 3202, and F(ω) is the transfer function of thefeedback network.

In embodiments, expected functions for Δt_(pk)(ω) may be specifiedwithin the protocol specifications for communication protocols used. Forexample, for 10 Gigabit Ethernet, this is specified in clause 52.8 ofthe IEEE 802.3 standard.

In embodiments, the system of FIG. 3B generally includes FPGA 3100 whichpreferably includes FPGA transceiver banks 3102. In embodiments, theFPGA transceiver banks 3102 includes deserializer 3104 on the receiverend of FPGA 3100, adjustable transceiver PLL 3108, and serializer 3110on the transmitter end of FPGA 3100. Deserializer 3104 may be similar tothe deserializer 3104′ and deserializer 3104″ discussed below inconnection with FIGS. 6A-6E, the descriptions of which applying herein.Serializer 3110 may be similar to serializer 3110′ and serializer 3110″discussed below in connection with FIGS. 7A and 7B respectively, thedescriptions of which applying herein. In embodiments, deserializer 3104receives a serial data stream from the I/O module 3120 and providesdeserialized data to the FPGA core 3106. More specifically, thedeserializer 3104 will provide parallel data streams to the FPGA core3106 for processing. As illustrated, this parallel data is provided tothe computational logic 3210 where processing and computation takesplace. FIG. 3B illustrates computational logic 3210 as a single elementwith a receiver side and a transmitting side. In embodiments, thecomputational logic 3210 may be implemented as a plurality of logicelements. In embodiments, when all computation is complete, theresulting data may be provided to serializer 3110 and then transmittedout of FPGA 3100, preferably via I/O module 3120, for example. Inembodiments, the I/O module may be a direct soldered cable, on-chipoptics, or an on-board optical transceiver, to name a few. Additionally,the I/O module may be an SFP, a QSFP, a micro QSFP, or a QSFP-DD, toname a few.

In embodiments, the serializer 3110 and deserializer 3104 (SERDES)circuit(s) may contain two major sections: (1) an analog side whosepurpose is signal cleaning, and (2) a digital side which turns theanalog signal into bits and converts between parallel and serial datastreams. In embodiments, the analog side may include a few differenttypes of amplifiers in order to provide signal cleaning.

In embodiments, as shown in FIG. 3B, an adjustable transceiver PLL 3108may be located inside the FPGA 3100 fabric with adjustable feedbackdividers, which in turn adjust the frequency of the PLL. Alternatively,the adjustable transceiver PLL 3108 may comprise a phase adjuster on theoutput of the loop, which adjusts the phase of the PLL directly. Unlikethe PLL with phase adjustment 3300 in FIG. 3A, the adjustabletransceiver PLL 3108 in FIG. 3B is located outside the FPGA core 3106and within the FPGA transceiver banks 3102. The adjustable transceiverPLL 3108 in FIG. 3B may also differ from the PLL with phase adjustment3300 in FIG. 3A in operating frequency.

As illustrated in FIG. 3B, in embodiments, the first receiver side clocksignal RXCLOCK is based on, but has a different frequency and phasethan, the REFERENCE CLOCK signal provided by oscillator 3000 based onthe operation of the deserializer 3104, as is discussed above withrespect to the serializer 104. In embodiments, the transmitter sideclock signal TXCLOCK provided from the serializer 3110 may be providedby adjustable transceiver phase locked loop 3108. In this manner, thefrequency or phase of the transmitter side clock signal TXCLOCK may beadjusted.

In embodiments, an internal phase controller 3202 is connected to thephase adjustment interface of the phase-controlled transceiver PLL 3108and provides control signals to allow for adjustment of the phase of atleast the transmitter side clock signal TXCLOCK. In embodiments, thephase of the incoming data may be adjusted by adjusting the data streamreceived by I/O module 3120. In embodiments, the internal phasecontroller 3202 provides control signals based on the phase differencebetween the receiver clock signal RXCLOCK and the transmitter clocksignal TXCLOCK. In embodiments, the phase difference may be determinedusing phase detector 3206.

Using the configuration of FIG. 3B, the receiver clock signal RXCLOCKand the transmitter clock signal TXCLOCK may be sufficiently aligned inphase such that there is no need for the clock domain crossing circuitdiscussed above, eliminating a technical problem. The technical solutionto a technical problem, in embodiments, is shown in the system of FIG.3B, which provides a wire or a logic path via the controller 3202 andadjustable transceiver PLL 3108 to adjust the phase of the transmitterside clock TXCLOCK to correspond to that of the first receiver sideclock signal RXCLOCK. Some delays may be introduced in the pathincluding min_delay and max_delay constraints of the wire andcomponents. However, such delays are insignificant compared to thelarger delays that are necessarily present when an asynchronous clockdomain crossing circuit is used.

In embodiments, with a phase difference between about 45° and 360°, themin_delay should be set at 0 and the max_delay should be set equal tothe time between rising edges on the receive clock signal and transmitclock signal (as shown in FIGS. 2A and 2B). The difference between 45°and 360° approximates the minimum amount of time required to safelycapture data. In embodiments, the formula for the time is as follows:t _(min) =t _(setup) +t _(wire) +t _(clock-out) Δt _(jitter) +t _(skew)

Referencing the above equation, t_(setup) is the setup time of aflip-flop (i.e., a latch) on FPGA 3100. The t_(wire) is the delay ofthe, for example, shortest wire (as compared to other wires on the FPGA)on FPGA 3100. In embodiments, the shortest wire is the shortest possiblewire. In embodiments, t_(clock-out) is the clock-to-output time of aflip-flop on the FPGA 3100. In embodiments, Δt_(jitter) is a safetyfactor for jitter on both clocks. Jitter, for example, is the deviationfrom true periodicity of a periodic signal from a reference clock. Inembodiments, t_(skew) is the skew between clock paths to the twoflip-flops on FPGA 3100.

In embodiments where there is a phase difference with the transmitterclock running less than 45° ahead of the receiver clock, metastabilitymay occur unless the clocks are treated as being greater than 360degrees apart (as seen in FIG. 2C). In such embodiments, the min and maxdelay are set with the min delay greater than the time differencebetween rising edges and the max delay less than one full clock cycleplus the min delay. In embodiments, the transmit clock signal may alsosample on the opposite edge as the receiving clock signal launch edge,but cases are symmetric. In embodiments, a change from positive tonegative edge sampling adds an approximate 180 degrees offset. Inembodiments, manufacturer analysis tools may add a reasonable jittermargin to account for times when jitter causes temporary misalignment.

In the embodiment of FIG. 3B, to avoid instability, the internal PLL ofthe adjustable transceiver PLL 3108 preferably runs at the highestbandwidth available from the PLL on the FPGA, while the overall phaselocking system including the phase detectors 3206 and controller 3202runs at a low bandwidth. In embodiments, the highest bandwidth availablefrom the PLL on the FPGA may range between 100 kHz and 10 MHz. In suchembodiments, the poles in the adjustable transceiver PLL 3108 areprevented from affecting the gain or phase margin of the off-chip PLLwhich may be included in the controller 3202. In embodiments, any loopfilter order may be used in the controller 3202 as long as the bandwidthis low enough to avoid interaction of poles in the control system. Inembodiments, a low bandwidth may be, for example, 100 Hz-40 kHz, 400 Hzand 4 khz, 4 kHz and 10 kHz, 100 hz to 40 kHz, 0 Hz and 100 Hz, to namea few. In embodiments, a second-order loop filter may be used in aneffort to improve lock time and phase locking error. In embodiments athird-order filter may be used if the second-order filter is stable. Inembodiments, a first-order filter is also an option when thesecond-order filter has stability problems. In embodiments, other orderfilters may be used, depending on the internal bandwidth of the FPGA3100.

In embodiments, where the bandwidth of the controller 3202 is too low,the receiving clock and the transmitter clock may not track each otherclosely enough to consider them locked. In embodiment, such trackingissues may be solved by adjusting the min and max delay constraintsappropriately to provide a cushion for the clocks being away from theexpected phase. In embodiments, a PD (Phase Detector) control system maybe provided after the loop filter to aid in resolving tracking issues.In embodiments, tracking issues may be addressed by adding somenonlinearity in the transfer function of the filter. Other solutions mayalso be possible.

In the embodiment of FIG. 3B, the controller 3202 utilizes aphase-locked loop. In embodiments, a derivative component may beprovided in the loop filter to slow down the control loop when theoutput value has changed recently and reduces the chance that the PLLson the FPGA will lose lock. In embodiments, the derivative component'scoefficient will be negative (i.e., the derivative component will actagainst the loop filter, however, some references refer to this as beinga positive direction) and less than 1 (so the path straight from thefilter dominates). The loop filter consists of a standard PLL loopfilter with bandwidth between 400 Hz and 40 kHz of up to 4th order plusa derivative component that slows down the loop based on recent changesin the filter value. The bandwidth of the standard PLL loop filter, insome embodiments, may range between 400 Hz and 4 khz, 4 kHz and 10 kHz,100 hz to 40 kHz, 0 Hz and 100 Hz, to name a few. The extra derivativecomponent helps to keep the other components in the control system intheir linear operating regions. The loop filter is represented byDigitalFilter( ) in the pseudocode provided below.

FIG. 3C illustrates a block diagram of another embodiment of an FPGAsystem in accordance with an exemplary embodiment of the presentinvention. Like in the embodiment shown in FIG. 3A, a PLL with phaseadjustment 3300 a located within the FPGA core 3106 is used by the FPGAsystem in FIG. 3C. Unlike in the embodiment shown in FIG. 3A, however,an additional control loop, comprising a phase detector 3206 b, acontroller 3202 b, and an external delay element 3300 b, may be used toallow multiple deserializers 3104 a, 3104 b and serializers 3110 a, 3110b to be aligned. Multiple deserializers and serializers, as opposed toone of each, allow FPGA 3100 to receive more input data and more outputdata. In such systems, for every additional receiver that needs to bephase-aligned, an additional control loop is required.

In embodiments, the additional control loop may use a similar method tothe control loop for phase aligning a transmitter to the receiver.However, in embodiments, the control loop may use a delay element 3300 bon the received data rather than adjusting the frequency or phase of anoscillator. The control loop may also include a deserializer 3104 b inthe feedback path instead of a transceiver PLL. In such embodiments, therequired bandwidth may change or cause different types ofnonlinearities. Otherwise, the method for phase-aligning additionalreceivers generally corresponds to the method used to phase align atransmitter and a receiver, as discussed.

In embodiments, the external delay element 3300 b may have an adjustmentrange at least as wide as one period of the parallel RX clock. This canbe accomplished by using a long delay line or a pair of smaller delaylines with a glitch-free switchover circuit to hide the boundaryconditions of the delay lines. In embodiments, the delay element 3300 bmay be implemented with a digital delay line (e.g., the SY89295U fromMicrel), or a voltage-controlled delay element (e.g., the HMC910 fromAnalog devices).

In embodiments, the additional controller 3202 b utilizes a delay-lockedloop. In embodiments, a derivative component may be provided in the loopfilter to slow down the control loop when the output value has changedrecently and reduces the chance that the PLLs on the FPGA will loselock. In embodiments, the derivative component's coefficient will benegative, i.e., the derivative component will act against the loopfilter (however, some references refer to this as being a “positive”direction) and less than 1 so that the path straight from the filterdominates. The loop filter consists of a standard PLL loop filter withbandwidth between 400 Hz and 40 kHz of up to 4th order plus a derivativecomponent that slows down the loop based on recent changes in the filtervalue. In embodiments, the bandwidth of the standard PLL loop filter mayrange between 400 Hz and 4 kHz, 4 kHz and 10 kHz, 100 Hz and 40 kHz, 0Hz and 100 Hz, to name a few. The extra derivative component helps tokeep the other components in the control system in their linearoperating regions. The loop filter is represented by DigitalFilter( ) inthe pseudocode provided below.

In the embodiment of FIG. 3C, there are three clocks which should bealigned, and the required timing constraints for passing data from oneclock domain to another all follow the same pattern. In embodimentswhere there is a phase difference between any pair of aligned clockswith one clock running less than 45 degrees ahead of the other clock,metastability may occur unless the clocks are treated as being greaterthan 360 degrees apart (as seen in FIG. 2C). In such embodiments, themin and max delay are set with the min delay greater than the timedifference between rising edges and the max delay less than one fullclock cycle plus the min delay. In embodiments, one of the clock signalsmay also sample on the opposite edge as the signal launch edge, butcases are symmetric. In embodiments, a change from positive to negativeedge sampling adds an approximate 180 degrees offset. In embodiments,manufacturer analysis tools may add a reasonable jitter margin toaccount for times when jitter causes temporary misalignment.

FIG. 3D illustrates a block diagram of another embodiment of an FPGAsystem in accordance with an exemplary embodiment of the presentinvention. Like in the embodiment shown in FIG. 3B, an adjustabletransceiver PLL 3108 located within the FPGA transceiver banks 3102 isused by the FPGA system of FIG. 3D. Unlike in the embodiment shown inFIG. 3B, however, an additional control loop similar to the additionalcontrol loop in FIG. 3C is used to allow multiple deserializers 3104 a,3104 b and serializers 3110 a, 3110 b to be aligned. In embodiments, theadditional control loop may operate in the manner identical to theadditional control loop described for FIG. 3C.

FIG. 3E illustrates a block diagram of another embodiment of an FPGAsystem in accordance with an exemplary embodiment of the presentinvention. This embodiment is similar to the embodiment shown in FIG. 3Cin almost all aspects, except that the delay element 3300 b is internalto the FPGA 3100 in FIG. 3E, unlike the delay element 3300 b locatedoutside of the FPGA 3100 in FIG. 3C. In embodiments, the delay element3300 b in FIG. 3E has the same requirements as the external delayelement shown in FIG. 3C.

FIG. 3F illustrates a block diagram of another embodiment of an FPGAsystem in accordance with an exemplary embodiment of the presentinvention. This embodiment is similar to the embodiment shown in FIG. 3Din almost all aspects, except that the delay element 3300 b is internalto the FPGA 3100 in FIG. 3F, unlike the delay elements 3300 b locatedoutside of the FPGA 3100 in FIG. 3D. The delay element 3300 b has thesame requirements as the external delay element shown in FIG. 3D.

FIG. 4A is a block diagram of another FPGA system in accordance with anexemplary embodiment of the present invention. The system of FIG. 4A issimilar to that of FIG. 3A or 3B, and common components are referred tousing the same reference numbers (with a differing first digit toindicate the Figure—e.g., if the first digit of a reference numeral is“3,” the reference numeral is located on FIG. 3A or 3B) as in FIGS. 3Aand 3B. One difference between the system of FIG. 4A and the system ofFIGS. 3A and 3B is that the phase detector 4206 is off-chip, i.e., notof the FPGA 4100. Another difference between the system of FIG. 4A andthe system of FIGS. 3A and 3B is that the FPGA 4100 includes zero-delaybuffer PLLS 4208 a, 4208 b. Another difference between the system ofFIG. 4A and the system of FIGS. 3A and 3B is that an internal phasecontroller 4202 is operationally connected to frequency or phaseadjustable oscillator 4200, which is outside the FPGA 4100, such that aphase or frequency control signal may be transmitted from the internalphase controller 4202 to transceiver PLL 4108 via the frequency orphase-adjustable oscillator 4200.

Referring back to FIG. 4A, in embodiments, the system generally includesFPGA 4100 which preferably includes FPGA transceiver banks 4102. Inembodiments, the FPGA transceiver banks 4102 includes deserializer 4104on the receiver end of FPGA 4100, transceiver PLL 4108, and serializer4110 on the transmitter end of FPGA 4100. Deserializer 4104 may besimilar to the deserializer 3014′ and deserializer 3104″ discussed belowin connection with FIGS. 6A-6E, the descriptions of which applyingherein. Serializer 4110 may be similar to serializer 3110′ andserializer 3110″ discussed below in connection with FIGS. 7A and 7Brespectively, the descriptions of which applying herein. In embodiments,deserializer 4104 receives a serial data stream via a first plurality ofdata pins of a first interface from the I/O module 4120 and providesdeserialized data to the FPGA core 4106. More specifically, thedeserializer 4104 will provide parallel data streams to the FPGA core4106 for processing. As illustrated, this parallel data is provided toreceiver side computational element 4210 a where processing andcomputation takes place. The resulting data may then be provided totransmitting side computational element 4210 b where additionalcomputation and processing may take place. FIG. 4A illustratescomputational logic 4210 as a single element. In embodiments, thecomputational logic 4210 may be implemented as a plurality of logicelements. In embodiments, computational logic 4210 may also include oneor more of the following: RAM blocks or dedicated hardened accelerators.In embodiments, when all computation is complete, the resulting data maybe provided to serializer 4110 and then transmitted out of FPGA 4100,preferably via I/O module 4120, for example. In embodiments, the I/Omodule 4120 may be a direct soldered cable, on-chip optics, or anon-board optical transceiver, to name a few. I/O Module 4120, may be,for example, a Samtec® I/O Module Firefly® I/O Module, to name a few.Additionally, the I/O module may be designed to meet one or more of thefollowing standards: SFP, a QSFP, a micro QSFP, or a QSFP-DD, to name afew.

In embodiments, FPGA 4100 may include one or more interfaces, such aspins. By way of example, a first interface of the one or moreinterfaces, may include a first plurality of pins of FPGA 4100. Thefirst plurality of pins may be used to transmit and/or receive dataand/or signals. In embodiments, the first plurality of pins may includea first reference clock pin. The first reference clock pin may beoperationally connected to deserializer 4104. In embodiments, the firstreference clock pin may also be operationally connected tofixed-frequency oscillator 4000 such that a first clock signal having afirst phase and a first phase may be transmitted from fixed-frequencyoscillator 4000 to deserializer 4104 via the first reference clock pin.In embodiments, the first clock signal may have a frequency between 50and 800 MHZ.

In embodiments, the first plurality of pins may also include a secondreference clock pin. The second reference clock pin may, in embodiments,be operationally connected to transceiver PLL 4108. The second referenceclock pin, in embodiments, may be operationally connected to frequencyor phase adjustable oscillator 4200 such that a second clock signalhaving a second phase and a second frequency may be transmitted from thefrequency or phase-adjustable oscillator 4200 to transceiver PLL 4108via the second reference clock pin. In embodiments, the second referenceclock pin may be operationally connected to serializer 4110 such thatthe second clock signal may be transmitted from the frequency orphase-adjustable oscillator 4200 to serializer 4110 via the secondreference clock pin. In embodiments, the second reference clock pin mayalso receive a first wire rate clock signal.

In embodiments, the first plurality of pins may also include a firstplurality of data pins. The first plurality of data pins may beoperationally connected deserializer 4104. In embodiments, the firstplurality of data pins may also be operationally connected to I/O module4120 such that a first serial data stream may be transmitted fromexternal connection 4122 to deserializer via the I/O module 4120 and thefirst plurality of data pins. In embodiments, the first serial datastream may range between 10 to 25 GBPS.

In embodiments, the first plurality of pins may also include a secondplurality of data pins. The second plurality of data pins may beoperationally connected to serializer 4110. In embodiments, the secondplurality of data pins may also be operationally connected to I/O module4120 such that a second serial data stream may be transmitted fromserializer 4110 to the external connection 4122 via the I/O module 4120and the second plurality of data pins. In embodiments, the second serialdata stream may range between 10 to 25 GBPS.

Although reference is made to separate first reference clock pin, secondreference clock pin, first plurality of data pins and second pluralityof data pins, in embodiments, each pin may be a fixed-function,fixed-location pin, connected internally through a wire. In embodiments,each pin may be a multiplexed fixed-function pin, connected internallythrough a multiplexer or crossbar. In embodiments, each pin may be ageneral purpose I/O pin connected through the FPGA core 4106. Inembodiments, one or more pin may be fixed function, fixed locationpines, while other pins may me multiplexed fixed function pins and/or ageneral purpose I/O pin.

In embodiments, the one or more interfaces may also include at least asecond interface. The second interface of the one or more interfaces,may include a second plurality of pins of FPGA 4100. The secondplurality of pins may be used to transmit and/or receive data and/orsignals. In embodiments, the second plurality of pins may include afirst clock output pin. The first clock output pin may be operationallyconnected to deserializer 4104. In embodiments, the first clock outputpin may also be operationally connected to computation logic 4210 suchthat a first receiver side clock signal is transmitted from thedeserializer 4104 to computation logic 4210 via the first clock outputpin. Additionally, in some embodiments, the first clock output pin mayalso be operationally connected to zero-delay buffer PLL 4208 a suchthat the receiver side clock is transmitted from the deserializer 4104to the zero-delay buffer PLL 4208 a via the first clock output pin. Inembodiments the first receiver side clock signal may have a frequencyranging between 100-650 MHZ.

In embodiments, the second plurality of pins may also include a secondclock output pin. The second clock output pin may be operationallyconnected to serializer 4110. In embodiments, the second clock outputpin may also be operationally connected to computation logic 4210 suchthat a transmitter side clock signal is transmitted from the serializer4110 to computation logic 4210 via the second clock output pin.Additionally, in some embodiments, the second clock output pin may alsobe operationally connected to zero-delay buffer PLL 4208 b such that thetransmitter side clock is transmitted from the serializer 4110 to thezero-delay buffer PLL 4208 b via the second clock output pin. Inembodiments the transmitter side clock signal may have a frequencyranging between 100-650 MHZ.

In embodiments, the serializer 4110 and deserializer 4104 (SERDES)circuit(s) may contain two major sections: (a) an analog side whosepurpose is signal cleaning, and (2) a digital side which turns theanalog signal into bits and converts between parallel and serial datastreams. In embodiments, the analog side may include a few differenttypes of amplifiers in order to provide signal cleaning. In embodiments,the adjustable oscillator 4200 may be implemented in a variety of ways.In embodiments, the adjustable oscillator 4200 may be implemented as avoltage controlled oscillator. A voltage controller oscillator may beimplemented using a variety of architectures. In embodiments, othertypes of oscillators may be used including negative-resistanceoscillators, Clapp oscillators, Colpitts oscillators, ring oscillators,and varactor-tuned oscillators, to name a few.

In embodiments, a voltage controller crystal oscillator may be used asthe adjustable oscillator 4200, for example, the Si550 from SiliconLabs. In embodiments, a numerically/digitally-controlled oscillator maybe used as the adjustable oscillator 4200, which is a digital version ofan analog VCO, and may use switched circuit elements or a fixedfrequency oscillator and a digital PLL to adjust the frequency.

In embodiments, the adjustable oscillator 4200 may be implemented with adigital delay line, e.g., the SY89295U from Micrel.

In embodiments, the adjustable oscillator 4200 may be implemented as avoltage-controlled delay element, e.g., the HMC910 provided by Analogdevices.

As illustrated in FIG. 4A, in embodiments, the first receiver side clocksignal RXCLOCK is based on but has a different frequency and phase thanthe REFERENCE CLOCK signal provided by oscillator 4200 based on theoperation of the deserializer 4104, as is discussed above with respectto the serializer 4110. In embodiments, the transmitter side clocksignal TXCLOCK provided from the deserializer 4104 may be provided bytransceiver phase locked loop 4108, which is provided inside FPGA 4100,and may be controlled by frequency or phase adjustable oscillator 4200,external to FPGA 4100. In this manner, the frequency or phase of thetransmitter side clock signal TXCLOCK may be adjusted.

In embodiments, an internal phase controller 4202 is connected to thefrequency or phase adjustable oscillator 4200 and provides controlsignals to allow for adjustment of the phase of at least the transmitterside clock signal TXCLOCK. The phase of the incoming data, inembodiments, may be adjusted by adjusting the data stream received byI/O module 4120. In embodiments, the internal phase controller 4202provides control signals based on the phase difference between thereceiver clock signal RXCLOCK and the transmitter clock signal TXCLOCK.In embodiments, the phase difference may be determined using phasedetector 4206 and zero-delay buffers, 4208 a, 4208 b. In embodiments,the zero-delay buffers 4208 a, 4208 b may be incorporated into the FPGAcore 4106 of the FPGA 4100. In embodiments, FPGA core 4106 may notinclude the zero-delay buffers 4208 a, 4208 b. In embodiments,zero-delay buffer 4208 a may have a reference trace T1 whose length ismatched to a length of wire T3 between the I/O pin of FPGA 4100 andexternal phase detector 4206. Similarly, zero delay buffer 4208 b mayhave a reference trace T2 whose length is matched to a second length ofwire T4 between the I/O pin of the FPGA 4100 and the external phasedetector 4206. Any length mismatch between reference traces T1 and wireT3 between the I/O pin of the FPGA 4100 and the external phase detector4206 may introduce a deterministic phase error proportional to themismatch of the length of the wires. Any length mismatch betweenreference traces T2 and wire T4 between the I/O pin of the FPGA 4100 andthe external phase detector 4206 may introduce a deterministic phaseerror proportional to the mismatch of the length of the wires.

In embodiments, the transceiver PLL 4108 may include a second adjustableoscillator. In embodiments, the second adjustable oscillator of thetransceiver PLL 4108 may provide the wire rate signal to the serializer4110 which may be used to provide the transmitter clock signal TX CLOCK.In embodiments, the adjustable oscillator of the transceiver PLL 4108may be provided outside of the transceiver PLL 4108 and operativelyconnected to the serializer 4110. In embodiments, where a secondadjustable oscillator is provided in the FPGA 4100, the adjustableoscillator 4200 may not be necessary and the output of the controller4202 may be provided to and used by the second adjustable oscillator toadjust the first wire rate clock signal W.R. CLOCK provided to theserializer 4110 based on the information from the controller. Inembodiments, where the second adjustable oscillator is provided on theFPGA 4100 and the adjustable oscillator 4200 is also provided off theFPGA, the delay of the second adjustable oscillator may be set to aconstant value and the controller 4202 may provide instructions to thesecond adjustable oscillator based on this constant value. Inembodiments, the constant value may be zero.

Using the configuration of FIG. 4A, the receiver clock signal RXCLOCKand the transmitter clock signal TXCLOCK are sufficiently aligned inphase such that there is no need for the clock domain crossing circuitdiscussed above, eliminating a technical problem. The technical solutionto a technical problem, in embodiments, is shown in the system of FIG.4A, which provides a wire or a logic path via the controller 4202 andadjustable oscillator 4200 to adjust the phase of the transmitter sideclock TMCLOCK to correspond to that of the first receiver side clocksignal RXCLOCK. Some delays may be introduced in the path includingmin_delay and max_delay constraints of the wire and components, however,such delays are insignificant compared to the larger delays that arenecessarily present when an asynchronous clock domain crossing circuitis used.

In embodiments, zero delay buffers 4208 a, 4208 b may be configured toadd a phase offset such that the receiver clock signal RXCLOCK and thetransmitter clock signal TXCLOCK may be provided with any desired fixedphase offset with respect to each other. In such embodiments, any phasedifference may be used, depending on the length of the logic pathbetween the clocks and the corresponding delay constraints. Inembodiments, with a phase difference between about 45° and 360°, themin_delay should be set at 0 and the max_delay should be set equal tothe time between rising edges on the receive clock signal and transmitclock signal (as shown in FIGS. 2A and 2B). The difference between 45°and 360° approximates the minimum amount of time required to safelycapture data. In embodiments, the formula for the time is as follows:t _(min) =t _(setup) +t _(wire) +t _(clock-out) +Δt _(jitter) +t _(skew)

Referencing the above equation, t_(setup) is the setup time of aflip-flop (i.e., a latch) on FPGA 4100. The t_(wire) is the delay ofthe, for example, shortest wire (as compared to other wires on the FPGA)on FPGA 4100. In embodiments, the shortest wire is the shortest possiblewire. In embodiments, t_(clock-out) is the clock-to-output time of aflip-flop on the FPGA 4100. In embodiments, Δt_(jitter) is a safetyfactor for jitter on both clocks. Jitter, for example, is the deviationfrom true periodicity of a periodic signal from a reference clock. Inembodiments, t_(skew) is the skew between clock paths to the twoflip-flops on FPGA 4100.

In embodiments where there is a phase difference with the transmitterclock running less than 45 degrees ahead of the receiver clock,metastability may occur unless the clocks are treated as being greaterthan 360 degrees apart (as seen in FIG. 2C). In such embodiments, themin and max delay are set with the min delay greater than the timedifference between rising edges and the max delay less than one fullclock cycle plus the min delay. In embodiments, the transmit clocksignal may also sample on the opposite edge as the receiving clocksignal launch edge, but cases are symmetric. In embodiments, a changefrom positive to negative edge sampling adds an approximate 180 degreesoffset. In embodiments, manufacturer analysis tools may add a reasonablejitter margin to account for times when jitter causes temporarymisalignment.

In embodiments, to avoid instability, both the transceiver PLL 4108 andthe zero-delay buffer PLL 4208 b preferably run at the highest bandwidthavailable, while the overall phase locking system including the phasedetectors 4206, controller 4202 and oscillator 4200 runs at a lowbandwidth. In such embodiments, the poles in the on-chip PLLs on theFPGA 4100 are prevented from affecting the gain or phase margin of theoff-chip PLL which may be included in the controller 4202. Inembodiments, any loop filter order may be used in the controller 4202 aslong as the bandwidth is low enough to avoid interaction of poles in thecontrol system. In embodiments, a low bandwidth may be, for example, 100Hz-40 kHz, 400 Hz and 4 khz, 4 kHz and 10 kHz, 100 hz to 40 kHz, 0 Hzand 100 Hz, to name a few. In embodiments, a second-order loop filtermay be used in an effort to improve lock time and phaselocking error. Inembodiments a third-order filter may be used if the second-order filteris stable. In embodiments, a first-order filter is also an option whenthe second-order filter has stability problems. In embodiments, otherorder filters may be used, depending on the internal bandwidth of theFPGA 4100.

In embodiments, where the bandwidth of the controller 4202 is too low,the receiving clock and the transmitter clock may not track each otherclosely enough to consider them locked. In embodiment, such trackingissues may be solved by adjusting the min and max delay constraintsappropriately to provide a cushion for the clocks being away from theexpected phase. In embodiments, a PD control system may be providedafter the loop filter to aid in resolving tracking issues. Inembodiments, tracking issues may be addressed by adding somenonlinearity in the transfer function of the filter. Other solutions mayalso be possible.

In the embodiment of FIG. 4A, the controller 4202 utilizes aphase-locked loop. In embodiments, a derivative component may beprovided in the loop filter to slow down the control loop when theoutput value has changed recently and reduces the chance that the PLLson the FPGA will lose lock. In embodiments, the derivative component'scoefficient will be negative (i.e., the derivative component will actagainst the loop filter, however, some references refer to this as beinga positive direction) and less than 1 (so the path straight from thefilter dominates). The loop filter consists of a standard PLL loopfilter with bandwidth between 400 Hz and 40 kHz of up to 4th order plusa derivative component that slows down the loop based on recent changesin the filter value. The bandwidth of the standard PLL loop filter, insome embodiments, may range between 400 Hz and 4 khz, 4 kHz and 10 kHz,100 hz to 40 kHz, 0 Hz and 100 Hz, to name a few. The extra derivativecomponent helps to keep the other components in the control system intheir linear operating regions. The loop filter is represented byDigitalFilter( ) in the pseudocode provided below.

FIG. 4B illustrates a block diagram of another embodiment of an FPGAsystem in accordance with an exemplary embodiment of the presentinvention. Like the embodiment shown in FIG. 4A, the embodiment of FIG.4B uses a frequency or phase adjustable oscillator 4200 a locatedoutside the FPGA 4100. Unlike in the embodiment shown in FIG. 4A, anadditional control loop, comprising a phase detector 4206 b, acontroller 4202 b, and an external delay element 4200 b, is used in theembodiment of FIG. 4B to allow multiple deserializers 4104 a, 4104 b andserializers 4110 a, 4110 b to be aligned. For every additional receiverthat needs to be phase-aligned, an additional control loop is required.

In embodiments, the external delay element 4200 b may have an adjustmentrange at least as wide as one period of the parallel RX clock. This canbe accomplished by using a long delay line or a pair of smaller delaylines with a glitch-free switchover circuit to hide the boundaryconditions of the delay lines. In embodiments, the delay element 4200 bmay be implemented with a digital delay line (e.g., the SY89295U fromMicrel), or a voltage-controlled delay element (e.g., the HMC910 fromAnalog devices).

In embodiments, the additional controller 4202 b utilizes a delay-lockedloop. In embodiments, a derivative component may be provided in the loopfilter to slow down the control loop when the output value has changedrecently and reduces the chance that the PLLs on the FPGA will loselock. In embodiments, the derivative component's coefficient will benegative, i.e., the derivative component will act against the loopfilter (noting, however, that some references refer to this as being a“positive” direction) and less than 1 so that the path straight from thefilter dominates. The loop filter consists of a standard PLL loop filterwith bandwidth between 400 Hz and 40 kHz of up to 4th order plus aderivative component that slows down the loop based on recent changes inthe filter value. The bandwidth of the standard PLL loop filter, in someembodiments, may range between 400 Hz and 4 kHz, 4 kHz and 10 kHz, 100Hz and 40 kHz, 0 Hz and 100 Hz, to name a few. The extra derivativecomponent helps to keep the other components in the control system intheir linear operating regions. The loop filter is represented byDigitalFilter( ) in the pseudocode provided below.

In the embodiment of FIG. 4B, there may be three clocks which should bealigned, and the required timing constraints for passing data from oneclock domain to another all follow the same pattern. In embodimentswhere there is a phase difference between any pair of aligned clockswith one clock running less than 45 degrees ahead of the other clock,metastability may occur unless the clocks are treated as being greaterthan 360 degrees apart (as seen in FIG. 2C). In such embodiments, themin and max delay are set with the min delay greater than the timedifference between rising edges and the max delay less than one fullclock cycle plus the min delay. In embodiments, one of the clock signalsmay also sample on the opposite edge as the signal launch edge, butcases are symmetric. In embodiments, a change from positive to negativeedge sampling adds an approximate 180 degrees offset. In embodiments,manufacturer analysis tools may add a reasonable jitter margin toaccount for times when jitter causes temporary misalignment.

Similar to the system described in connection with FIG. 4A, the systemof FIG. 4B may or may not include zero delay buffers 4208 a, 4208 b, and4208 c. In embodiments, zero-delay buffer 4208 a may have a referencetrace T1 whose length is matched to a length of wire T4 between the I/Opin of FPGA 4100 and external phase detectors 4206 a, 4206 b. Similarly,zero delay buffer 4208 b may have a reference trace T2 whose length ismatched to a second length of wire T5 between the I/O pin of the FPGA4100 and the external phase detector 4206 a. Also, similarly, zero delaybuffer 4208 c may have a reference trace T3 whose length is matched to asecond length of wire T6 between the I/O pin of the FPGA 4100 and theexternal phase detector 4206 b. Any length mismatch between referencetraces T1 and wire T4 between the I/O pin of the FPGA 4100 and theexternal phase detectors 4206 a, 4206 b may introduce a deterministicphase error proportional to the mismatch of the length of the wires. Anylength mismatch between reference traces T2 and wire T5 between the I/Opin of the FPGA 4100 and the external phase detector 4206 a mayintroduce a deterministic phase error proportional to the mismatch ofthe length of the wires. Any length mismatch between reference traces T3and wire T6 between the I/O pin of the FPGA 4100 and the external phasedetector 4206 b may introduce a deterministic phase error proportionalto the mismatch of the length of the wires.

FIG. 4C illustrates a block diagram of another embodiment of an FPGAsystem in accordance with an exemplary embodiment of the presentinvention. This embodiment is similar to the embodiment shown in FIG. 4Bin almost all aspects, except that the delay element 4200 b is internalto the FPGA 4200 b, unlike the delay element 4200 b located outside ofthe FPGA 4100 in FIG. 4B. In embodiments, the delay element 4200 b inFIG. 4C has the same requirements as the external delay element shown inFIG. 4B.

FIG. 5A is a block diagram of another FPGA system in accordance with anexemplary embodiment of the present invention. The system of FIG. 5A issimilar to that of FIG. 4A, and common components are referred to usingthe same reference numbers (with a differing first digit to indicate theFigure—e.g., if the first digit of a reference numeral is “4,” thereference numeral is located on FIG. 4A) as in FIG. 4A. Like in thesystem of FIG. 4A, the phase detector 5206 is off-chip, i.e., not of theFPGA 5100, and the FPGA 5100 also includes zero-delay buffer PLLS 5208a, 5208 b. On the other hand, the system of FIG. 5A is different fromthe system of FIG. 4A in that an internal phase controller 5202 isoperationally connected to the PLL with phase adjustment 5300 locatedwithin the FPGA core 5106, through which the internal phase controllerprovides phase control to the transceiver PLL 5108.

FIG. 5A illustrates a block diagram of an FPGA system in accordance withan exemplary embodiment of the present disclosure that provides phasecontrol while minimizing latency. In embodiments, the system generallyincludes FPGA 5100 which preferably includes FPGA transceiver banks5102. In embodiments, the FPGA transceiver banks 5102 includesdeserializer 5104 on the receiver end of FPGA 5100, transceiver PLL5108, and serializer 5110 on the transmitter end of FPGA 5100.Deserializer 5104 may be similar to the deserializer 3104′ anddeserializer 3104″ discussed below in connection with FIGS. 6A-6E, thedescriptions of which applying herein. Serializer 5110 may be similar toserializer 3110′ and serializer 3110″ discussed below in connection withFIGS. 7A and 7B respectively, the descriptions of which applying herein.In embodiments, deserializer 5104 receives a serial data stream from theI/O module 5120 and provides deserialized data to the FPGA core 5106.More specifically, the deserializer 5104 will provide parallel datastreams to the FPGA core 5106 for processing. As illustrated, thisparallel data is provided to the computational logic 5210 whereprocessing and computation takes place. FIG. 5A illustratescomputational logic 5210 as a single element with a receiver side and atransmitting side. In embodiments, the computational logic 5210 may beimplemented as a plurality of logic elements. In embodiments, when allcomputation is complete, the resulting data may be provided toserializer 5110 and then transmitted out of FPGA 5100, preferably viaI/O module 5120, for example. In embodiments, the I/O module may be adirect soldered cable, on-chip optics, or an on-board opticaltransceiver, to name a few. Additionally, the I/O module may be an SFP,a QSFP, a micro QSFP, or a QSFP-DD, to name a few.

In embodiments, the serializer 5110 and deserializer 5104 (SERDES)circuit(s) may contain two major sections: (1) an analog side whosepurpose is signal cleaning, and (2) a digital side which turns theanalog signal into bits and converts between parallel and serial datastreams. In embodiments, the analog side may include a few differenttypes of amplifiers in order to provide signal cleaning.

In embodiments, as shown in FIG. 5A, a PLL with phase adjustment 5300may be located inside the FPGA 5100 fabric with adjustable feedbackdividers, which in turn adjust the frequency of the PLL. Alternatively,the PLL with phase adjustment 5300 may comprise a phase adjuster on theoutput of the loop, which adjusts the phase of the PLL directly. Inembodiments, the same description may apply to adjustable transceiverPLLs 3108 and 5108 respectively shown in FIGS. 3B and 5B, but thedifferences are the location of the PLL inside the FPGA (e.g., withinthe FPGA core 5106 versus outside the FPGA core and within the FPGAtransceiver banks 3102, 5102) and the operating frequency.

As illustrated in FIG. 5A, in embodiments, the first receiver side clocksignal RXCLOCK is based on but has a different frequency and phase thanthe REFERENCE CLOCK signal provided by oscillator 5000 based on theoperation of the deserializer 5104, as is discussed above with respectto the serializer 104. In embodiments, the transmitter side clock signalTXCLOCK provided from the serializer 5110 may be provided by transceiverphase locked loop 5108, which is provided inside FPGA 5100, and may becontrolled by PLL with Phase Adjustment 5300 of the FPGA core 5106. Inthis manner, the frequency or phase of the transmitter side clock signalTXCLOCK may be adjusted.

As illustrated in FIG. 5A, in embodiments, the first receiver side clocksignal RXCLOCK is based on but has a different frequency and phase thanthe REFERENCE CLOCK signal provided by oscillator 5000 based on theoperation of the deserializer 5104, as is discussed above with respectto the serializer 104. In embodiments, the transmitter side clock signalTXCLOCK provided from the serializer 5110 may be provided by transceiverphase locked loop 5108, which is provided inside FPGA 5100, and may becontrolled by a PLL with Phase Adjustment 5300 of the FPGA core 5106. Inthis manner, the frequency or phase of the transmitter side clock signalTXCLOCK may be adjusted.

In embodiments, an internal phase controller 5202 is connected to thePLL with Phase Adjustment 5300 and provides control signals to allow foradjustment of the phase of at least the transmitter side clock signalTXCLOCK. In embodiments, the phase of the incoming data may be adjustedby adjusting the data stream received by I/O module 5120. Inembodiments, the phase difference may be determined using phase detector5206 which is external to the FPGA. In embodiments, with a phasedifference between about 45° and 360°, the min_delay should be set at 0and the max_delay should be set equal to the time between rising edgeson the receive clock signal and transmit clock signal (as shown in FIGS.2A and 2B). The difference between 45° and 360° approximates the minimumamount of time required to safely capture data. In embodiments, theformula for the time is as follows:t _(min) =t _(setup) +t _(wire) +t _(clock-out) Δt _(jitter) +t _(skew)

Referencing the above equation, t_(setup) is the setup time of aflip-flop (i.e., a latch) on FPGA 3100. The t_(wire) is the delay ofthe, for example, shortest wire (as compared to other wires on the FPGA)on FPGA 3100. In embodiments, the shortest wire is the shortest possiblewire. In embodiments, t_(clock-out) is the clock-to-output time of aflip-flop on the FPGA 3100. In embodiments, Δt_(jitter) is a safetyfactor for jitter on both clocks. Jitter, for example, is the deviationfrom true periodicity of a periodic signal from a reference clock. Inembodiments, t_(skew) is the skew between clock paths to the twoflip-flops on FPGA 3100.

In embodiments where there is a phase difference with the transmitterclock running less than 45 degrees ahead of the receiver clock,metastability may occur unless the clocks are treated as being greaterthan 360 degrees apart (as seen in FIG. 2C). In such embodiments, themin and max delay are set with the min delay greater than the timedifference between rising edges and the max delay less than one fullclock cycle plus the min delay. In embodiments, the transmit clocksignal may also sample on the opposite edge as the receiving clocksignal launch edge, but cases are symmetric. In embodiments, a changefrom positive to negative edge sampling adds an approximate 180 degreesoffset. In embodiments, manufacturer analysis tools may add a reasonablejitter margin to account for times when jitter causes temporarymisalignment.

In the embodiment of FIG. 5A, both the transceiver PLL 5108 and thezero-delay buffer PLL 5208 b preferably run at the highest bandwidthavailable from the PLL on the FPGA, while the overall phase lockingsystem including the phase detectors 5206, controller 5202 and PLL withPhase Adjustment 5300 runs at a low bandwidth. In such embodiments, thepoles in the adjustable transceiver PLL 5108 are prevented fromaffecting the gain or phase margin of the off-chip PLL which may beincluded in the controller 5202. In embodiments, the highest bandwidthavailable from the PLL on the FPGA may range between 100 kHz and 10 MHz.In embodiments, any loop filter order may be used in the controller 5202as long as the bandwidth is low enough to avoid interaction of poles inthe control system. In embodiments, a low bandwidth may be, for example,100 Hz-40 kHz, 400 Hz and 4 kHz, 4 kHz and 10 kHz, 100 Hz and 40 kHz, 0Hz and 100 Hz, to name a few. In embodiments, a second-order loop filtermay be used in an effort to improve lock time and phaselocking error. Inembodiments a third-order filter may be used if the second-order filteris stable. In embodiments, a first-order filter is also an option whenthe second-order filter has stability problems. In embodiments, otherorder filters may be used, depending on the internal bandwidth of theFPGA 5100.

In embodiments, the one or more interfaces may also include at least asecond interface. The second interface of the one or more interfaces,may include a second plurality of pins of FPGA 5100. The secondplurality of pins may be used to transmit and/or receive data and/orsignals. In embodiments, the second plurality of pins may include afirst clock output pin. The first clock output pin may be operationallyconnected to deserializer 5104. In embodiments, the first clock outputpin may also be operationally connected to computation logic 5210 suchthat a first receiver side clock signal is transmitted from thedeserializer 5104 to computation logic 5210 via the first clock outputpin. Additionally, in some embodiments, the first clock output pin mayalso be operationally connected to zero-delay buffer PLL 5208 a suchthat the receiver side clock is transmitted from the deserializer 5104to the zero-delay buffer PLL 5208 a via the first clock output pin. Inembodiments the first receiver side clock signal may have a frequencyranging between 100-650 MHZ.

In embodiments, the second plurality of pins may also include a secondclock output pin. The second clock output pin may be operationallyconnected to the serializer 5110. In embodiments, the second clockoutput pin may also be operationally connected to computation logic 5210such that a transmitter side clock signal is transmitted from theserializer 5110 to computation logic 5210 via the second clock outputpin. Additionally, in some embodiments, the second clock output pin mayalso be operationally connected to zero-delay buffer PLL 5208 b suchthat the transmitter side clock is transmitted from the serializer 5110to the zero-delay buffer PLL 5208 b via the second clock output pin. Inembodiments the transmitter side clock signal may have a frequencyranging between 100-650 MHZ.

The PLL with internal phase adjustment 5300 may be similar to the PLL3300 in FIG. 3A. It is a component of the FPGA core fabric 5106 whichgenerates a clock of adjustable phase and frequency.

In embodiments, where the bandwidth of the controller 5202 is too low,the receiving clock and the transmitter clock may not track each otherclosely enough to consider them locked. In embodiment, such trackingissues may be solved by adjusting the min and max delay constraintsappropriately to provide a cushion for the clocks being away from theexpected phase. In embodiments, a PD (Phase Detector) control system maybe provided after the loop filter to aid in resolving tracking issues.In embodiments, tracking issues may be addressed by adding somenonlinearity in the transfer function of the filter. Other solutions mayalso be possible.

In the embodiment of FIG. 5A, the controller 5202 utilizes aphase-locked loop. In embodiments, a derivative component may beprovided in the loop filter to slow down the control loop when theoutput value has changed recently and reduces the chance that the PLLson the FPGA will lose lock. In embodiments, the derivative component'scoefficient will be negative (i.e., the derivative component will actagainst the loop filter, however, some references refer to this as beinga positive direction) and less than 1 (so the path straight from thefilter dominates). The loop filter consists of a standard PLL loopfilter with bandwidth between 400 Hz and 40 kHz of up to 4th order plusa derivative component that slows down the loop based on recent changesin the filter value. The bandwidth of the standard PLL loop filter, insome embodiments, may range between 400 Hz and 4 khz, 4 kHz and 10 kHz,100 hz to 40 kHz, 0 Hz and 100 Hz, to name a few. The extra derivativecomponent helps to keep the other components in the control system intheir linear operating regions. The loop filter is represented byDigitalFilter( ) in the pseudocode provided below.

FIG. 5B is a block diagram of another FPGA system in accordance with anexemplary embodiment of the present invention. Unlike in the embodimentshown in FIG. 5A, an internal phase controller 5202 in the system ofFIG. 5B is directly connected and provides phase control signals to theadjustable transceiver PLL 5108, which is located in the FPGAtransceiver banks 5102 and outside the FPGA core 5106.

FIG. 5B illustrates a block diagram of an FPGA system in accordance withan embodiment of the present disclosure that provides phase controlwhile minimizing latency wherein the FPGA has an acceptablephase-controlled transceiver PLL 5108. This embodiment uses thephase-controlled transceiver PLL directly to control the phase ofTXCLOCK. An acceptable phase-control system must have adjustment gaincapable of matching phase transients on the incoming data stream towithin the timing margin (t_(min)), satisfying the following inequality:

∫₀^(∞)Δ t_(p k)(ω)[1 − LF(ω)] d ω ≤ t_(min)

Where Δt_(pk)(ω) is the peak design jitter at that frequency and LF isthe transfer function over frequency of the phase locking systemconsisting of the phase detector 5206, controller 5202, and PLL 5108.Expected functions for Δt_(pk)(ω) are specified within the protocolspecifications for communication protocols used. For 10 GigabitEthernet, this is specified in clause 52.8 of the IEEE 802.3 standard.

In embodiments, the system of FIG. 5B generally includes FPGA 5100 whichpreferably includes FPGA transceiver banks 5102. In embodiments, theFPGA transceiver banks 5102 includes deserializer 5104 on the receiverend of FPGA 5100, transceiver PLL 5108, and serializer 5110 on thetransmitter end of FPGA 5100. Deserializer 5104 may be similar to thedeserializer 3104′ and deserializer 3104″ discussed below in connectionwith FIGS. 6A-6E, the descriptions of which applying herein. Serializer5110 may be similar to serializer 3110′ and serializer 3110″ discussedbelow in connection with FIGS. 7A and 7B respectively, the descriptionsof which applying herein. In embodiments, deserializer 5104 receives aserial data stream from the I/O module 5120 and provides deserializeddata to the FPGA core 5106. More specifically, the deserializer 5104will provide parallel data streams to the FPGA core 5106 for processing.As illustrated, this parallel data is provided to the computationallogic 5210 where processing and computation takes place. FIG. 5Billustrates computational logic 5210 as a single element with a receiverside and a transmitting side. In embodiments, the computational logic5210 may be implemented as a plurality of logic elements. Inembodiments, when all computation is complete, the resulting data may beprovided to serializer 5110 and then transmitted out of FPGA 5100,preferably via I/O module 5120, for example. In embodiments, the I/Omodule may be a direct soldered cable, on-chip optics, or an on-boardoptical transceiver, to name a few. Additionally, the I/O module may bean SFP, a QSFP, a micro QSFP, or a QSFP-DD, to name a few.

In embodiments, the serializer 5110 and deserializer 5104 (SERDES)circuit(s) may contain two major sections: (1) an analog side whosepurpose is signal cleaning, and (2) a digital side which turns theanalog signal into bits and converts between parallel and serial datastreams. In embodiments, the analog side may include a few differenttypes of amplifiers in order to provide signal cleaning.

In embodiments, as shown in FIG. 5B, an adjustable transceiver PLL 5108may be located inside the FPGA 5100 fabric with adjustable feedbackdividers, which in turn adjust the frequency of the PLL. Alternatively,the adjustable transceiver PLL 5108 may comprise a phase adjuster on theoutput of the loop, which adjusts the phase of the PLL directly. Unlikethe PLL with phase adjustment 5300 in FIG. 5A, the adjustabletransceiver PLL 5108 in FIG. 5B is located outside the FPGA core 5106and within the FPGA transceiver banks 5102. The adjustable transceiverPLL 5108 in FIG. 5B may also differ from the PLL with phase adjustment5300 in FIG. 5A in operating frequency.

As illustrated in FIG. 5B, in embodiments, the first receiver side clocksignal RXCLOCK is based on, but has a different frequency and phasethan, the REFERENCE CLOCK signal provided by oscillator 5000 based onthe operation of the deserializer 5104, as is discussed above withrespect to the serializer 104. In embodiments, the transmitter sideclock signal TXCLOCK provided from the serializer 5110 may be providedby adjustable transceiver phase locked loop 5108. In this manner, thefrequency or phase of the transmitter side clock signal TXCLOCK may beadjusted.

In embodiments, an internal phase controller 5202 is connected to thephase adjustment interface of the phase-controlled transceiver PLL 5108and provides control signals to allow for adjustment of the phase of atleast the transmitter side clock signal TXCLOCK. The phase of theincoming data, in embodiments, may be adjusted by adjusting the datastream received by I/O module 5120. In embodiments, the internal phasecontroller 5202 provides control signals based on the phase differencebetween the receiver clock signal RXCLOCK and the transmitter clocksignal TXCLOCK. In embodiments, the phase difference may be determinedusing phase detector 5206.

Using the configuration of FIG. 5B, the receiver clock signal RXCLOCKand the transmitter clock signal TXCLOCK may be sufficiently aligned inphase such that there is no need for the clock domain crossing circuitdiscussed above, eliminating a technical problem. The technical solutionto a technical problem, in embodiments, is shown in the system of FIG.5B, which provides a wire or a logic path via the controller 5202 andadjustable transceiver PLL 5108 to adjust the phase of the transmitterside clock TXCLOCK to correspond to that of the first receiver sideclock signal RXCLOCK. Some delays may be introduced in the pathincluding min_delay and max_delay constraints of the wire andcomponents, however, such delays are insignificant compared to thelarger delays that are necessarily present when an asynchronous clockdomain crossing circuit is used.

In embodiments, with a phase difference between about 45⁰ and 360°, themin_delay should be set at 0 and the max_delay should be set equal tothe time between rising edges on the receive clock signal and transmitclock signal (as shown in FIGS. 2A and 2B). The difference between 45⁰and 360° approximates the minimum amount of time required to safelycapture data. In embodiments, the formula for the time is as follows:t _(min) =t _(setup) +t _(wire) +t _(clock-out) Δt _(jitter) +t _(skew)

Referencing the above equation, t_(setup) is the setup time of aflip-flop (i.e., a latch) on FPGA 5100. The t_(wire) is the delay ofthe, for example, shortest wire (as compared to other wires on the FPGA)on FPGA 5100. In embodiments, the shortest wire is the shortest possiblewire. In embodiments, t_(clock-out) is the clock-to-output time of aflip-flop on the FPGA 5100. In embodiments, Δt_(jitter) is a safetyfactor for jitter on both clocks. Jitter, for example, is the deviationfrom true periodicity of a periodic signal from a reference clock. Inembodiments, t_(skew) is the skew between clock paths to the twoflip-flops on FPGA 5100.

In embodiments where there is a phase difference with the transmitterclock running less than 45° ahead of the receiver clock, metastabilitymay occur unless the clocks are treated as being greater than 360degrees apart (as seen in FIG. 2C). In such embodiments, the min and maxdelay are set with the min delay greater than the time differencebetween rising edges and the max delay less than one full clock cycleplus the min delay. In embodiments, the transmit clock signal may alsosample on the opposite edge as the receiving clock signal launch edge,but cases are symmetric. In embodiments, a change from positive tonegative edge sampling adds an approximate 180 degrees offset. Inembodiments, manufacturer analysis tools may add a reasonable jittermargin to account for times when jitter causes temporary misalignment.

In the embodiment of FIG. 5B, both the transceiver PLL 5108 and thezero-delay buffer PLL 5208 b preferably run at the highest bandwidthavailable from the PLL on the FPGA, while the overall phase lockingsystem including the phase detectors 5206 and controller 5202 runs at alow bandwidth. In such embodiments, the poles in the adjustabletransceiver PLL 5108 are prevented from affecting the gain or phasemargin of the off-chip PLL which may be included in the controller 5202.In embodiments, the highest bandwidth available from the PLL on the FPGAmay range between 100 kHz and 10 MHz. In embodiments, any loop filterorder may be used in the controller 5202 as long as the bandwidth is lowenough to avoid interaction of poles in the control system. Inembodiments, a low bandwidth may be, for example, 100 Hz-40 kHz, 400 Hzand 4 kHz, 4 kHz and 10 kHz, 100 Hz and 40 kHz, 0 Hz and 100 Hz, to namea few. In embodiments, a second-order loop filter may be used in aneffort to improve lock time and phaselocking error. In embodiments athird-order filter may be used if the second-order filter is stable. Inembodiments, a first-order filter is also an option when thesecond-order filter has stability problems. In embodiments, other orderfilters may be used, depending on the internal bandwidth of the FPGA5100.

In embodiments, the one or more interfaces may also include at least asecond interface. The second interface of the one or more interfaces,may include a second plurality of pins of FPGA 5100. The secondplurality of pins may be used to transmit and/or receive data and/orsignals. In embodiments, the second plurality of pins may include afirst clock output pin. The first clock output pin may be operationallyconnected to deserializer 5104. In embodiments, the first clock outputpin may also be operationally connected to computation logic 5210 suchthat a first receiver side clock signal is transmitted from thedeserializer 5104 to computation logic 5210 via the first clock outputpin. Additionally, in some embodiments, the first clock output pin mayalso be operationally connected to zero-delay buffer PLL 5208 a suchthat the receiver side clock is transmitted from the deserializer 5104to the zero-delay buffer PLL 5208 a via the first clock output pin. Inembodiments the first receiver side clock signal may have a frequencyranging between 100-650 MHz.

In embodiments, the second plurality of pins may also include a secondclock output pin. The second clock output pin may be operationallyconnected to serializer 5110. In embodiments, the second clock outputpin may also be operationally connected to computation logic 5210 suchthat a transmitter side clock signal is transmitted from the serializer5110 to computation logic 5210 via the second clock output pin.Additionally, in some embodiments, the second clock output pin may alsobe operationally connected to zero-delay buffer PLL 5208 b such that thetransmitter side clock is transmitted from the serializer 5110 to thezero-delay buffer PLL 5208 b via the second clock output pin. Inembodiments the transmitter side clock signal may have a frequencyranging between 100-650 MHz.

In embodiments, where the bandwidth of the controller 5202 is too low,the receiving clock and the transmitter clock may not track each otherclosely enough to consider them locked. In embodiment, such trackingissues may be solved by adjusting the min and max delay constraintsappropriately to provide a cushion for the clocks being away from theexpected phase. In embodiments, a PD (Phase Detector) control system maybe provided after the loop filter to aid in resolving tracking issues.In embodiments, tracking issues may be addressed by adding somenonlinearity in the transfer function of the filter. Other solutions mayalso be possible.

In the embodiment of FIG. 5B, the controller 5202 utilizes aphase-locked loop. In embodiments, a derivative component may beprovided in the loop filter to slow down the control loop when theoutput value has changed recently and reduces the chance that the PLLson the FPGA will lose lock. In embodiments, the derivative component'scoefficient will be negative (i.e., the derivative component will actagainst the loop filter, however, some references refer to this as beinga positive direction) and less than 1 (so the path straight from thefilter dominates). The loop filter consists of a standard PLL loopfilter with bandwidth between 400 Hz and 40 kHz of up to 4th order plusa derivative component that slows down the loop based on recent changesin the filter value. The bandwidth of the standard PLL loop filter, insome embodiments, may range between 400 Hz and 4 khz, 4 kHz and 10 kHz,100 hz to 40 kHz, 0 Hz and 100 Hz, to name a few. The extra derivativecomponent helps to keep the other components in the control system intheir linear operating regions. The loop filter is represented byDigitalFilter( ) in the pseudocode provided below.

FIG. 5C illustrates a block diagram of another embodiment of an FPGAsystem in accordance with an exemplary embodiment of the presentinvention. Like the embodiment shown in FIG. 5A, a PLL with phaseadjustment 5300 a located within the FPGA core 5106 is used by the FPGAsystem in FIG. 5C. Unlike in the embodiment shown in FIG. 5A, anadditional control loop, comprising a phase detector 5206 b, acontroller 5202 b, and an external delay element 5300 b, is used toallow multiple deserializers 5104 a, 5104 b and serializers 5110 a, 5110b to be aligned. For every additional receiver that needs to bephase-aligned, an additional control loop is required.

In embodiments, the external delay element 5300 b may have an adjustmentrange at least as wide as one period of the parallel RX clock. This canbe accomplished by using a long delay line or a pair of smaller delaylines with a glitch-free switchover circuit to hide the boundaryconditions of the delay lines. In embodiments, the delay element 5200 bmay be implemented with a digital delay line (e.g., the SY89295U fromMicrel), or a voltage-controlled delay element (e.g., the HMC910 fromAnalog devices).

In embodiments, the additional controller 5202 b utilizes a delay-lockedloop. In embodiments, a derivative component may be provided in the loopfilter to slow down the control loop when the output value has changedrecently and reduces the chance that the PLLs on the FPGA will loselock. In embodiments, the derivative component's coefficient will benegative, i.e., the derivative component will act against the loopfilter (noting, however, that some references refer to this as being a“positive” direction) and less than 1 so the path straight from thefilter dominates. The loop filter consists of a standard PLL loop filterwith bandwidth between 400 Hz and 40 kHz of up to 4th order plus aderivative component that slows down the loop based on recent changes inthe filter value. In embodiments, the bandwidth of the standard PLL loopfilter may range between 400 Hz and 4 kHz, 4 kHz and 10 kHz, 100 Hz and40 kHz, 0 Hz and 100 Hz, to name a few. The extra derivative componenthelps to keep the other components in the control system in their linearoperating regions. The loop filter is represented by DigitalFilter( ) inthe pseudocode provided below.

Similar to the system described in connection with FIG. 5A, the systemof FIG. 5C may or may not include zero delay buffers 5208 a, 5208 b, and5208 c. In embodiments, zero-delay buffer 5208 a may have a referencetrace T1 whose length is matched to a length of wire T4 between the I/Opin of FPGA 4100 and external phase detectors 5206 a, 5206 b. Similarly,zero delay buffer 5208 b may have a reference trace T2 whose length ismatched to a second length of wire T5 between the I/O pin of the FPGA5100 and the external phase detector 5206 a. Also, similarly, zero delaybuffer 5208 c may have a reference trace T3 whose length is matched to asecond length of wire T6 between the I/O pin of the FPGA 5100 and theexternal phase detector 5206 b. Any length mismatch between referencetraces T1 and wire T4 between the I/O pin of the FPGA 5100 and theexternal phase detectors 5206 a, 5206 b may introduce a deterministicphase error proportional to the mismatch of the length of the wires. Anylength mismatch between reference traces T2 and wire T5 between the I/Opin of the FPGA 5100 and the external phase detector 5206 a mayintroduce a deterministic phase error proportional to the mismatch ofthe length of the wires. Any length mismatch between reference traces T3and wire T6 between the I/O pin of the FPGA 5100 and the external phasedetector 5206 b may introduce a deterministic phase error proportionalto the mismatch of the length of the wires.

In the embodiment of FIG. 5C, there may be three clocks which should bealigned, and the required timing constraints for passing data from oneclock domain to another all follow the same pattern. In embodimentswhere there is a phase difference between any pair of aligned clockswith one clock running less than 45 degrees ahead of the other clock,metastability may occur unless the clocks are treated as being greaterthan 360 degrees apart (as seen in FIG. 2C). In such embodiments, themin and max delay are set with the min delay greater than the timedifference between rising edges and the max delay less than one fullclock cycle plus the min delay. In embodiments, one of the clock signalsmay also sample on the opposite edge as the signal launch edge, butcases are symmetric. In embodiments, a change from positive to negativeedge sampling adds an approximate 180 degrees offset. In embodiments,manufacturer analysis tools may add a reasonable jitter margin toaccount for times when jitter causes temporary misalignment.

FIG. 5D illustrates a block diagram of another embodiment of an FPGAsystem in accordance with an exemplary embodiment of the presentinvention. Like in the embodiment shown in FIG. 5B, an adjustabletransceiver PLL 5108 located within the FPGA transceiver banks 5102 isused by the FPGA system of FIG. 3D. Unlike in the embodiment shown inFIG. 5B, an additional control loop similar to the additional controlloop in FIG. 5C is used to allow multiple deserializers 5104 a, 5104 band serializers 5110 a, 5110 b to be aligned. In embodiments, theadditional control loop in the embodiment of FIG. 5D operates in themanner identical to the additional control loop described for FIG. 5C.

FIG. 5E illustrates a block diagram of another embodiment of an FPGAsystem in accordance with an exemplary embodiment of the presentinvention. This embodiment is similar to the embodiment shown in FIG. 5Cin almost all aspects, except that the delay element 5300 b is internalto the FPGA 5100, unlike the delay element 5300 b located outside of theFPGA 5100 in FIG. 5C. In embodiments, the delay element 5300 b in FIG.5E has the same requirements as the external delay element 5300 b shownin FIG. 5C.

FIG. 5F illustrates a block diagram of another embodiment of an FPGAsystem in accordance with an exemplary embodiment of the presentinvention. This embodiment is similar to the embodiment shown in FIG. 5Din almost all aspects, except that the delay element 5300 b is internalto the FPGA 5100, unlike the delay element 5300 b located outside of theFPGA 5100 in FIG. 5D. In embodiments, the delay element 5300 b in FIG.5F has the same requirements as the external delay element 5300 b shownin FIG. 5D.

FIG. 6A is an exemplary block diagram of a deserializer suitable for usein the field programmable gate array systems of FIGS. 3A-3F, 4A-4C and5A-5F in accordance with an exemplary embodiment of the presentinvention. Deserializer 3104′, in embodiments, may include an inputbuffer 6104, a continuous time linear equalizer (CTLE) 6106, a variablegain amplifier (VGA) 6108; voltage adder 6110; decision feedbackequalizer (DFE) 6110, clock and data recovery (CDR) 6116, Serial toparallel box 6118, decision circuit 6112, and clock divider 6120. Inembodiments, deserializer 3104′ may not include one or more of thefollowing components: CTLE 6106, DFE 6114, and/or VGA 6108, to name afew.

A deserializer circuit accepts a data stream from an input buffer andconverts it into a parallel format at a lower bit rate. First, datacomes into the chip from the input pins 6102 into an input buffer 6104.In embodiments, the input buffer receives a transmission from an I/Omodule, such as I/O module 4120. A continuous time linear equalizer(CTLE) 6106 may be used after the input buffer 6104 to improve signalquality by placing a zero near the dominant pole of the transmissionmedium to cancel the first pole of the system. A variable-gain amplifier(VGA) 6108 may then be used to scale the signal up for furtherprocessing. This output is then turned into a binary 0-1 decision by thedecision circuit 6112. In embodiments, the decision circuit 6112includes adder 6110. A decision feedback amplifier may be included tofurther improve signal integrity by taking the results of the decisioncircuit, applying a weighting filter, and adding the result to theoutput from the VGA 6108. When the signal is in the digital domain, aclock and data recovery circuit (CDR) 6116 recovers the phase of thedata and create a wire-rate clock that is phase-aligned to the receiveddata. The CDR 6116 may use a reference clock and a priori knowledge ofthe nominal frequency of the data stream to set the frequency of therecovered clock. With a clock aligned to the data stream, which in someembodiments may be a high speed data stream (e.g., 10 Gbps, 25 Gbps, arange from 9-25 Gbps, 10-25 Gbps, 22-33 Gbps, 33-45 Gbps, 45-60 Gbps,60-80 Gbps, 80-120 Gbps, or 1 Gbps-100 Gbps, to name a few) the receivercan use a clock divider 6120 and a serial-to-parallel circuit 6118 toconvert the serial data stream into a lower-frequency than the datastream (e.g., ¼^(th) of the data rate of the high speed data stream or1/256^(th) of the data rate of the high speed data stream, to name afew) parallel data stream.

In embodiments, a decision feedback equalizer 6114 (DFE) may beprovided. In embodiments, the decision feedback equalizer may be afilter that subtracts or adds at least n decided bits to cancelinter-symbol interference on the wire. In embodiments, a clock and datarecovery circuit may be provided and includes a data phase detector,which may be any of the circuits discussed above, as well as aphase-locked loop circuit.

FIG. 6B is an exemplary circuit diagram of a continuous-time linearequalizer (CTLE) suitable for use in the deserializer of FIG. 6A inaccordance with an exemplary embodiment of the present invention. CTLE3106″, in some embodiments, places a zero to cancel out the dominantpole of the trans mission line for an incoming signal. In embodiments,CTLE 3106″ is a linear filter applied at a receiver, in the case of FIG.6B, input buffer 6104, that attenuates low-frequency signal componentsand filters off higher frequencies. The CTLE 3106″ may be adjusted tooptimize the ratio of low frequency attenuation to high frequencyamplification. In embodiments, a variable gain amplifier may be providedand used to scale the output of the CTLE to as large of a signal aspossible. In embodiments, a decision circuit may also be provided.

FIG. 6C is an exemplary block diagram of a clock and data recovery (CDR)circuit suitable for use in the deserializer of FIG. 6A in accordancewith an embodiment of the present invention. In embodiments, CDR 6116′circuit recovers phase information from an incoming data stream. Thedata stream, in embodiments, may be received by an alexander phasedetector 6302. The CDR 6116′ may also obtain frequency information froma reference clock, output. In embodiments, the output may determinewhether the recovered clock have the same phase as the recovered data.

FIG. 6D is an exemplary circuit diagram of a decision circuit 6112′suitable for use in the deserializer of FIG. 6A in accordance with anembodiment of the present invention. FIG. 6D illustrates an exemplaryembodiment of such a decision circuit 6112′ utilizing a StrongARM senseamplifier. In embodiments, any comparator circuit may be used as adecision circuit.

FIG. 6E illustrates an exemplary embodiment of deserializer 3104″. Inembodiments, the deserializer 3104″ may include a receiver PMA 6512 andreceiver PCS 6508. The receiver PMA 6512 may include receiver buffer6504, CDR 6516, and serial to parallel box 6118′. In embodiments, thereceiver buffer 6504 receives serial differential input data 6502. Inembodiments, this data is received by an I/O module, such as the I/Omodule 4120 of FIG. 4A. In embodiments, the data is then received by theCDR 6516 from the receiver buffer 6504. The CDR, in embodiments,transmits the inputted serial data, a serial clock signal, and aparallel clock signal to the serial to parallel box 6118′. The CDR mayalso transmits the parallel clock data to the receiver PCS 6508. Theserial to parallel box 6118′, in embodiments, then converts the serialdata into parallel data, transmitting the parallel data to the receiverPCS 6508. The receiver PCS 6508 transmits parallel data to the FPGA Core4106.

In addition to the components illustrated in FIG. 6E, deserializer 3104″may also include or be electrically coupled to a continuous-time linearequalizer (CTLE) (see FIG. 6B) which is a pure analog circuit that maybe used to correct channel issues. In embodiments, the deserializer3104″ may include a shift register or demultiplexer with a counter.

FIG. 7A is an exemplary block diagram of a serializer 3110′ suitable foruse in the field programmable gate array systems of FIGS. 3A-3F, 4A-4Cand 5A-5F in accordance with an exemplary embodiment of the presentinvention. The serializer circuit accepts a clock running at wire rate4112 (from transceiver PLL 4108) and accepts parallel data (e.g., TXdata 7102) to serialize the data. Data is converted from a parallelformat to a serial data stream in the parallel to serial box 7104. Afeed-forward equalizer (FFE) 7106, which uses the history of bitsserialized on the wire to filter out inter-symbol interference, may beincluded to improve signal integrity. Finally, the serializer outputsthe serial data stream to a transmission medium using an output buffer7114. The serializer also generates a parallel clock (TX Clk 7110), thatoperates at a lower speed than the data stream (e.g., 100-400 MHz,100-600 MHz, to name a few), for use by other components using either asimple clock divider 7108 or a more complex clock generation block whichcan contain dividers and phase shifting elements for aligning multipletransmitters.

FIG. 7B illustrates an exemplary embodiment of serializer 3110″. Asdiscussed above with respect to FIGS. 4A and 7A, serializer 3110″receives parallel data streams from the FPGA core 4106 and serializesthem for transmission out of the FPGA 4100. In embodiments, serializer3110″ may include transmitter PCS 7206, clock generation block 7210, andtransmitter PMA 7216. In embodiments, transmitter PMA 7216 may includeparallel to serial box 7104′ and transmitter buffer 7214. Thetransmitter PCS 7206 may receive parallel data from FPGA core 4106. Inembodiments, the parallel data is then transmitted to the parallel toserial box 7104′. The clock generation block 7210 may receive awire-rate clock signal 7112. The clock generation block 7210, inembodiments, may split the signal in to a parallel clock signal andserial clock signal, both the parallel and serial signals beingtransmitted to the parallel to serial box 7104′. The parallel to serialbox 7104′ may covert the data from parallel to serial, transmitting theserial differential output data 7218 to the transmitter buffer 7214.Then, in embodiments, the transmitter buffer 7214 outputs the serialdifferential output data 7218.

In embodiments the serializer 3110″ may be a shift register or amultiplexer with a counter. In embodiments, the serializer 3110″ mayinclude additional components, such as a feed forward equalizer (FFE)which uses the last n bits to cancel inter symbol interference on thewire. In embodiments, a buffer amplifier may be included which providesvariable gain to allow adjustable voltage swing on the output of thetransmitter. In embodiments, a clock divider may be provided. Inembodiments, the clock divider may generate a low-speed clock (e.g.,100-400 MHz, 100-600 MHz, to name a few) for the serializer and thePMA/PCS. In embodiments, the clock divider is typically provided using acounter. In embodiments, the clock generation block 7210 might alsoinclude a clock phase adjustment circuit to allow multiple transmitterlanes to be phase-aligned.

Referring to FIGS. 8A-8D, in embodiments, the phase detector 4206 may bean analog, single bit phase detector which may be implemented using avariety of circuit components including (1) an XOR gate; (2)S-Rflip-flop; (3) D flip-flop; (4) Edge-triggered JK flip-flop; (5) Gilbertcell multiplier and/or (6) Diode ring mixer, to name a few. FIG. 8Aillustrates an exemplary embodiment of a phase detector 3206′ thatutilizes D flip-flops. FIG. 8B illustrates another exemplary embodimentof a phase detector 3206″ that utilizes S-R flip-flops. FIG. 8C isanother exemplary block diagram of a phase detector 3206′″ suitable foruse in the field programmable gate array system of FIG. 4A in accordancewith an embodiment of the present invention. FIG. 8D is anotherexemplary block diagram of a phase detector 3206″″ suitable for use inthe field programmable gate array system of FIG. 4A in accordance withan embodiment of the present invention. Other than a basic phasedetector, a time-to-digital converter (TDC) can be used to detect thephase offset of the clocks. FIG. 8C shows a Vernier TDC and FIG. 8Dshows a parallel delay line TDC as examples. The Vernier TDC uses a pairof unequal delay lines to delay the two signals by varying amounts anduses D-type flip-flops to quantify the phase difference between the twosignals at multiple points along the delay line. The parallel delay lineTDC uses a set of simple phase detectors with the two input signalsdelayed by varying amounts to quantify the phase difference between thetwo signals. In both cases, a decoding circuit is used to translate theresults of the individual phase detection elements into a phase offset.

In embodiments, the phase detector 4206 may be a multi-bit phasedetector, sometimes referred to as a time-to digital converter (TDC). Inembodiments, such a phase detector may be a counter-based TDC thatcounts up the time between START and STOP. In embodiments, such a phasedetector may be an interpolator TDC in which time measurement may beperformed by measuring a voltage difference (e.g., by using a ramp waveand sampling at two points or by charging a capacitor). In embodiments,a phase detector may be a Vernier TDC as shown in FIG. 8C.

In embodiments the phase detector 4206 may be implemented using a tappeddelay line TDC. In embodiments, the phase detector 4206 may beimplemented using a metastability-based phase detector. In embodiments,the metastability-based phase detector is similar to a D flip-flop phasedetector but includes an averaging filter. In embodiments, aparallel-delay-line TDC may be used to implement the phase detector4206. FIG. 8D illustrates an example of a parallel-delay-line TDC. Inembodiments, the phase detector may be implemented as a pulse-shrinkingTDC. Such a pulse-shrinking TDC may generate a pulse and then send itcircularly through an engineered delay line to shrink it until it isgone, count the number of times it goes through the pulse-shrinkingcircuit.

In embodiments, the phase detector may be implemented as a scramblingTDC. Scrambling TDCs may be implemented with added digital noise shapingto suppress errors. In embodiments, the phase detector 4206 may beimplemented as a PLL/DLL-based TDC. In such an embodiment, the phasedetector may lock a digital PLL to the incoming feedback signal andrecord the internal control code.

In embodiments, all of the phase detector embodiments discussed abovemay be combined with or connected to a low-pass filter and an analog todigital converter (ADC). In embodiments, other examples of phasedetectors may be used consistent with the teachings of this disclosure.

The pseudo code shown below implements the control loop inside the PLLor DLL controller (e.g., 3202 in FIGS. 3A-3B, 4202 in FIG. 4A, or 5202in FIGS. 5A-5B) and communicates with the host in accordance with anexemplary embodiment of the present invention. In embodiments, thepseudocode shown below, or a similar pseudocode is run by the FPGA or asoft processor programmed on the FPGA and uses an interrupt- orevent-based system to run the control algorithm for either the PLL orDLL paths on the device. The main control loop is triggered on everysample of the internal ADC.

Begin:  Set up peripherals  Program initial configuration  Set up forperiodic free-running measurement from the TDC  Wait On PeriodicInterrupt:  Read TDC code into Window[0]  ControlCode =DigitalFilter(Window)  Read MODE pin to Mode  If Mode = PLL_MODE then:  Output ControlCode to PLL frequency adjustment interface  Else:  Output ControlCode to PLL phase adjustment interface  Shift Windowarray by 1

FIGS. 9A-C, 10A-B and 11 are flow charts of a process implementing thefield programmable gate array system in accordance with an exemplaryembodiment of the present invention. In embodiments, the processdescribed in the flow charts of FIGS. 9A-C, 10A-B and 11 may beimplemented by the field programmable gate array system shown in FIG.4A. The field programmable gate array system, including the fieldprogrammable gate array and the components thereof, may be similar tothe field programmable gate array systems of FIG. 4A, the descriptionsof which applying herein.

The process of FIGS. 9A-9C may begin at step S902. Referring to FIG. 9A,at step S902, a first serial data stream is received by a firstplurality of data pins of a first interface of a field programmable gatearray in the field programmable gate array system. In embodiments, thefirst serial data stream may include market data. In embodiments, marketdata may be price and trade-related data for a financial instrumentreported by a trading venue (i.e., a stock exchange).

At a step S904, a first clock signal having a first frequency and afirst phase is received by a first reference clock in in the firstinterface. In embodiments, step S904 may be performed before step S902.In embodiments step S904 may be performed contemporaneously with stepS902.

The process of FIGS. 9A-9C may continue with a step S906. At step S906,the first plurality of data pins of the first interface may transmit thefirst serial data stream to a deserializer in the field programmablegate array.

At a step S908, the first reference clock pin in the first interface maytransmit the first clock signal to the deserializer. In embodiments,step S908 may be performed before step S906. In embodiments step S908may be performed contemporaneously with step S906.

At a step S910, a first receiver side clock signal may be generated bythe deserializer. In embodiments, the first receiver side clock signalmay have a second frequency and a second phase. In embodiments, thesecond frequency is different from the first frequency. In embodiments,the second frequency corresponds to the first frequency. In embodiments,corresponding may refer to the frequencies being the same frequency. Inembodiments corresponding may also refer to frequencies being close(e.g., within an acceptable range in the art) to the same frequencies.In embodiments, the first phase and the second phase may not be aligned.In embodiments the first phase and the second phase are aligned. Inembodiments, a difference between the first phase and the second phaseis below a threshold level. In embodiments, the deserializer generatesthe first receiver clock signal based at least in part on the firstclock signal.

At a step S912, the first serial data stream received by thedeserializer may be converted, by the deserializer, into a firstplurality of data streams. In embodiments, the first plurality of datastreams may include a first amount of data streams. In embodiments, thefirst amount of data streams is equal to eight (8) data streams. Inembodiments, the first amount of data streams is equal to ten (10) datastreams. In embodiments, the first amount of data streams is equal tosixteen (16) data streams. In embodiments, the first amount of datastreams is equal to twenty (20) data streams. In embodiments, the firstamount of data streams is equal to thirty-two (32) data streams. Inembodiments, the first amount of data streams is equal to forty (40)data streams. In embodiments, the first amount of data streams is equalto sixty-four (64) data streams. In embodiments, the first amount ofdata streams is equal to eighty (80) data streams. In embodiments, thefirst amount of data streams is equal to one hundred twenty-eight (128)data streams. In embodiments, the first amount of data streams is equalto one hundred sixty (160) data streams. In embodiments, the firstamount of data streams is equal to two (2) to the power of N, where N isan integer. In embodiments, the first amount of data streams is equal toten (10) times two (2) to the power of N, where N is an integer. Inembodiments the first amount of data streams is equal to thirty-three(33) times two (2) to the power of N, where N is an integer.

At a step S914, the first receiver side clock signal is transmitted fromthe deserializer to computational circuitry in the field programmablegate array.

At a step S916, the first plurality of data streams is transmitted formthe deserializer to the computational circuitry. In embodiments, thefirst plurality of data streams includes a first plurality of dataitems. In embodiments, step S916 may be performed before step S914. Inembodiments step S916 may be performed contemporaneously with step S914.

At a step S918, the first receiver side clock signal is transmitted fromthe deserializer via a first clock output pin on a second interface ofthe field programmable gate array to a phase detector of the fieldprogrammable gate array system which is not on the field programmablegate array. In embodiments, the transmission of the first receiver sideclock signal is also via a first zero delay buffer phase lock loop ofthe field programmable gate array before being transmitted via the firstclock output pin of the second interface.

At a step S920, a first transmitter side clock signal is generated bythe field programmable gate array system. In embodiments, the firsttransmitter side clock signal has a third frequency and a third phase.In embodiments, the third frequency corresponds to the second frequency.In embodiments the third phase may be aligned with the second phase. Inembodiments, the third phase and the second phase may not be aligned. Inembodiments, the third phase and the second phase may have a differencein phases that is less than a second threshold level. In embodiments,the first transmitter side clock signal is generated by performing thefollowing steps until a first output of the phase detector is below afirst threshold.

Referring to FIG. 10A, generating the first transmitter side clocksignal may begin at step S1002. At step S1002, a second clock signal isgenerated by an adjustable oscillator in the field programmable gatearray system. In embodiments, the second clock signal may have a fourthfrequency and a fourth phase.

The process of FIGS. 10A-B may continue at a step S1004. At step S1004,a first wire rate clock signal having a fifth phase and a fifthfrequency is generated. In embodiments, the generation of the first wirerate clock signal may be based in part on the second clock signal. Inembodiments the fifth frequency is a rational multiple of the fourthfrequency. In embodiments the fifth frequency is equal to x times thefifth frequency where x is the second amount of data streams. Inembodiments the fifth frequency is equal to x times the frequency of theof the transmitter side clock signal where x is half of the secondamount of data streams.

In embodiments, the generation of the first wire rate clock signal maybe performed with the following steps. Referring to FIG. 11, at a stepS1102, the second clock signal is received from the adjustableoscillator at a transceiver phase lock loop via a second reference clockpin in the first interface. At a step S1104, the second clock signal isprocessed by the transceiver phase lock loop. As a step S1106, the firstwire rate clock signal is provided by the transceiver phase lock look.The providing of the first wire rate clock signal, in embodiments, maybe based at least in part on the processing of the received clocksignal. Once the first wire rate clock signal is provided, at a stepS1108. the first wire rate clock signal is transmitted by thetransceiver phase lock loop to a serializer on the field programmablegate array. In embodiments, the first wire rate clock is generated by aphase lock loop. In embodiments, the phase lock loop that generates thefirst wire rate clock signal is on field programmable gate array. Inembodiments, the phase lock loop that generates the first wire rateclock signal is part of the field programmable gate array system and notpart of the field programmable gate array.

In embodiments, the generation of the first wire rate clock signal mayinclude processing the second clock signal. Once the second clock signalis processed, the first wire rate clock signal may be provided. Theproviding of the first wire rate clock signal, in embodiments, is basedat least in part on the processing of the second clock signal. Afterproviding the first wire rate clock signal, in embodiments, the firstwire rate clock signal may be transmitted to the serializer.

Referring back to FIG. 10A, at a step 1006, an interim transmitter sideclock signal having a sixth frequency and a sixth phase is generated bythe serializer in the field programmable gate array. In embodiments, thesixth frequency corresponds to the third frequency. In embodiments thesixth phase may be aligned with the third phase. In embodiments, thesixth frequency may correspond to the third frequency and the sixthphase may not be aligned with the third phase. In embodiments, the sixthphase and the third phase may not be aligned. In embodiments, the thirdphase and the sixth phase may have a difference in phases that is lessthan a second threshold level.

At a step S1008, the interim transmitter side clock signal istransmitted from the serializer via a second clock output pin of thesecond interface to the phase detector. In embodiments, the transmissionof the interim transmitter side clock is also via a second zero delaybuffer phase lock loop of the field programmable gate array before beingtransmitted via the second clock output in of the second interface.

The process of FIGS. 10A-10B may continue with at a step S1010.Referring to FIG. 10B, at a step S1010, a first output is generated bythe phase detector. The first output, in embodiments, is based at leaston a comparison of the first receiver side clock signal and the interimside clock signal. The first output, in embodiments, may includeinformation indicating the phase differences (if there is any) betweenthe first receiver side clock and the interim side clock signal.Additionally, in embodiments, the first output may include informationindicating a difference (if there is any) of frequencies between thefirst receiver side clock signal and the interim side clock signal.

At a step S1012, the first output may be transmitted from the phasedetector to a phase controller of the field programmable gate arraysystem, which may be placed within the field programmable gate array.

At a step S1014, interim adjustment information is determined by thephase controller. In embodiments, the interim adjustment information maybe based at least in part on the first output. For example, the interimadjustment information may indicate necessary adjustments to align thephases of the first receiver side clock signal and the interim sideclock signal. As another example, the interim adjustment information mayindicate necessary adjustments to align the frequencies of the firstreceiver side clock signal and the interim side clock signal.

At a step S1016, the interim adjustment information is transmitted fromthe phase controller to the adjustable oscillator. In embodiments, oncethe interim adjustment information is received, the adjustableoscillator adjusts the second clock signal based on the interimadjustment information. The adjustment of the second clock signal, inembodiments, may change the phase of the second clock signal. Inembodiments, the interim adjustment information indicates a desiredphase. In embodiments, the interim adjustment information indicates adesired frequency. In embodiments, the interim adjustment informationindicates a change in phase. In embodiments, the interim adjustmentinformation indicates a change in frequency. In embodiments, theadjustment information includes a voltage. In embodiments, theadjustment information includes a digital transmission. For example, theadjustment information may be a command over a serial bus (e.g., I2C orSPI) that adjusts a clock divider or phase rotator. As another example,the adjustment information may trigger a set of parallel digital wiresfor “frequency/phase up,” “frequency/phase down,” and/or“frequency/phase step.” In embodiments, the adjustment of the secondclock signal may change the frequency of the second clock signal.

In embodiments, steps S1002 through steps S1016 are repeated until thefirst output of the phase detector is below the first threshold level.This threshold level may indicate that the first receiver side clocksignal and the interim transmitter side clock signal have phases thatare aligned. This threshold level may indicate that the first receiverside clock signal and the interim transmitter side clock signal havefrequencies that correspond to one another.

Referring back to FIG. 9B, once the first output of the phase detectoris below a first threshold level, at a step S922, the serializer maytransmit the first transmitter side clock signal to the computationalcircuitry. At a step S924, a first set of operations performed by thecomputational circuitry on at least a portion of the first plurality ofdata items to generate a second plurality of data streams. Inembodiments, the first operation includes a trading algorithm. Inembodiments the trading algorithm may include the parsing the marketdata. Once the market data is parsed, mathematical operations areperformed at a portion of the market data. After the mathematicaloperations are performed, order packets are generated using at least anoutput of the mathematical operations. In embodiments, the first set ofoperations includes at least one of the following: (i) an arithmeticoperation; (ii) a logical operation; (iii) a pipeline operation; and(iv) a memory access operation.

In embodiments, at least a portion of the first set of operations isperformed prior to step S922. In embodiments, at least a portion of thefirst set of operations is performed after step S922. In embodiments allof the first set of operations is performed after step S922.

In embodiments, the second plurality of data streams may include asecond amount of data streams. In embodiments, the second amount of datastreams may be the same as the first amount of data streams. Inembodiments, the second amount of data streams is equal to eight (8)data streams. In embodiments, the second amount of data streams is equalto ten (10) data streams. In embodiments, the second amount of datastreams is equal to sixteen (16) data streams. In embodiments, thesecond amount of data streams is equal to twenty (20) data streams. Inembodiments, the second amount of data streams is equal to thirty-two(32) data streams. In embodiments, the second amount of data streams isequal to forty (40) data streams. In embodiments, the second amount ofdata streams is equal to sixty-four (64) data streams. In embodiments,the second amount of data streams is equal to eighty (80) data streams.In embodiments, the second amount of data streams is equal to onehundred twenty-eight (128) data streams. In embodiments, the secondamount of data streams is equal to one hundred sixty (160) data streams.In embodiments, the second amount of data streams is equal to two (2) tothe power of N, where N is an integer. In embodiments, the second amountof data streams is equal to ten (10) times two (2) to the power of N,where N is an integer. In embodiments the second amount of data streamsis equal to thirty-three (33) times two (2) to the power of N, where Nis an integer.

At a step S926, the second plurality of data streams is transmitted fromthe computational circuitry to the serializer. Once received by theserializer, at a step S928, the serializer converts the second pluralityof parallel data streams into the second serial data stream. The secondserial data stream, in embodiments, includes trading data. Trading data,for example, may be any data related to purchasing or selling of stocks,commodities, goods, and/or services.

At a step S930, the second serial data stream may be transmitted fromthe serializer off the field gate programmable array system via a secondplurality of data pins of the first interface. In embodiments, thesecond serial data stream is transmitted from the serializer on thefield programmable gate array to an input/output module of the fieldgate array system but not on the field gate array. The second serialdata stream may then be transmitted from the input/output module off thefield gate array system.

FIGS. 12A-C, 13A-B and 14 are exemplary flow charts of a processimplementing the field programmable gate array system in accordance withanother exemplary embodiment of the present invention. In embodiments,the process described in the flow charts of FIGS. 12A-C, 13A-B and 14may be implemented by the field programmable gate array system shown inFIG. 3A or FIG. 5A. The field programmable gate array system, includingthe field programmable gate array and the components thereof, may besimilar to the field programmable gate array systems of FIGS. 3A and 5A,the descriptions of which applying herein.

The process of FIGS. 12A-12C may begin at step S1202. Referring to FIG.12A, at step S1202, a first serial data stream is received by a firstplurality of data pins of a first interface of a field programmable gatearray in the field programmable gate array system. In embodiments, thefirst serial data stream may include market data. In embodiments, marketdata may be price and trade-related data for a financial instrumentreported by a trading venue (i.e., a stock exchange).

At a step S1204, a first clock signal having a first frequency and afirst phase is received by a first reference clock in in the firstinterface. In embodiments, step S1204 may be performed before stepS1202. In embodiments, step S1204 may be performed contemporaneouslywith step S1202.

The process of FIGS. 12A-12C may continue with a step S1206. At stepS1206, the first plurality of data pins of the first interface maytransmit the first serial data stream to a deserializer in the fieldprogrammable gate array.

At a step S1208, the first reference clock pin in the first interfacemay transmit the first clock signal to the deserializer. In embodiments,step S1208 may be performed before step S1206. In embodiments step S1208may be performed contemporaneously with step S1206.

At a step S1210, a first receiver side clock signal may be generated bythe deserializer. In embodiments, the first receiver side clock signalmay have a second frequency and a second phase. In embodiments, thesecond frequency is different from the first frequency. In embodiments,the second frequency corresponds to the first frequency. In embodiments,corresponding may refer to the frequencies being the same frequency. Inembodiments corresponding may also refer to frequencies being close(e.g., within an acceptable range in the art) to the same frequencies.In embodiments, the first phase and the second phase may not be aligned.In embodiments the first phase and the second phase are aligned. Inembodiments, a difference between the first phase and the second phaseis below a threshold level. In embodiments, the deserializer generatesthe first receiver clock signal based at least in part on the firstclock signal.

At a step S1212, the first serial data stream received by thedeserializer may be converted, by the deserializer, into a firstplurality of data streams. In embodiments, the first plurality of datastreams may include a first amount of data streams. In embodiments, thefirst amount of data streams is equal to eight (8) data streams. Inembodiments, the first amount of data streams is equal to ten (10) datastreams. In embodiments, the first amount of data streams is equal tosixteen (16) data streams. In embodiments, the first amount of datastreams is equal to twenty (20) data streams. In embodiments, the firstamount of data streams is equal to thirty-two (32) data streams. Inembodiments, the first amount of data streams is equal to forty (40)data streams. In embodiments, the first amount of data streams is equalto sixty-four (64) data streams. In embodiments, the first amount ofdata streams is equal to eighty (80) data streams. In embodiments, thefirst amount of data streams is equal to one hundred twenty-eight (128)data streams. In embodiments, the first amount of data streams is equalto one hundred sixty (160) data streams. In embodiments, the firstamount of data streams is equal to two (2) to the power of N, where N isan integer. In embodiments, the first amount of data streams is equal toten (10) times two (2) to the power of N, where N is an integer. Inembodiments the first amount of data streams is equal to thirty-three(33) times two (2) to the power of N, where N is an integer.

At a step S1214, the first receiver side clock signal is transmittedfrom the deserializer to computational circuitry in the fieldprogrammable gate array.

At a step S1216, the first plurality of data streams is transmitted formthe deserializer to the computational circuitry. In embodiments, thefirst plurality of data streams includes a first plurality of dataitems. In embodiments, step S1216 may be performed before step S1214. Inembodiments step S1216 may be performed contemporaneously with stepS1214.

At a step S1218, the first receiver side clock signal is transmittedfrom the deserializer to a phase detector of the field programmable gatearray system. The phase detector may or may not be placed on the fieldprogrammable gate array. If the phase detector is not on the fieldprogrammable gate array, the first receiver side clock signal istransmitted from the deserializer to the phase detector via a firstclock output pin on a second interface of the field programmable gatearray. In embodiments, the transmission of the first receiver side clocksignal to the phase detector may also be via a first zero delay bufferphase lock loop of the field programmable gate array before beingtransmitted via the first clock output pin of the second interface.

At a step S1220, a first transmitter side clock signal is generated bythe field programmable gate array system. In embodiments, the firsttransmitter side clock signal has a third frequency and a third phase.In embodiments, the third frequency corresponds to the second frequency.In embodiments the third phase may be aligned with the second phase. Inembodiments, the third phase and the second phase may not be aligned. Inembodiments, the third phase and the second phase may have a differencein phases that is less than a second threshold level. In embodiments,the first transmitter side clock signal is generated by performing thefollowing steps until a first output of the phase detector is below afirst threshold.

Referring to FIG. 13A, generating the first transmitter side clocksignal may begin at step S1302. At step S1302, a second clock signal isgenerated by a phase lock loop with phase adjustment in the fieldprogrammable gate array system. In embodiments, the phase lock loop withphase adjustment is on the field programmable gate array. Inembodiments, the phase lock loop with phase adjustment is located withinthe core of the field programmable gate array. In embodiments, thesecond clock signal may have a fourth frequency and a fourth phase.

The process of FIGS. 13A-B may continue at a step S1304. At step S1304,a first wire rate clock signal having a fifth phase and a fifthfrequency is generated. In embodiments, the generation of the first wirerate clock signal may be based in part on the second clock signal. Inembodiments the fifth frequency is a rational multiple of the fourthfrequency. In embodiments the fifth frequency is equal to x times thefifth frequency where x is the second amount of data streams. Inembodiments the fifth frequency is equal to x times the frequency of theof the transmitter side clock signal where x may be half of the secondamount of data streams.

In embodiments, the generation of the first wire rate clock signal maybe performed with the following steps. Referring to FIG. 14, at a stepS1402, the second clock signal is received from the phase lock loop withphase adjustment at a transceiver phase lock loop via a second referenceclock pin in the first interface. At a step S1404, the second clocksignal is processed by the transceiver phase lock loop. As a step S1406,the first wire rate clock signal is provided by the transceiver phaselock loop. The providing of the first wire rate clock signal, inembodiments, may be based at least in part on the processing of thereceived second clock signal. Once the first wire rate clock signal isprovided, at a step S1408, the first wire rate clock signal istransmitted by the transceiver phase lock loop to a serializer on thefield programmable gate array. In embodiments, the first wire rate clocksignal is generated by a phase lock loop. In embodiments, the phase lockloop that generates the first wire rate clock signal is on fieldprogrammable gate array. In embodiments, the phase lock loop thatgenerates the first wire rate clock signal may be part of the fieldprogrammable gate array system and not part of the field programmablegate array.

In embodiments, the generation of the first wire rate clock signal mayinclude processing the second clock signal. Once the second clock signalis processed, the first wire rate clock signal may be provided. Theproviding of the first wire rate clock signal, in embodiments, is basedat least in part on the processing of the second clock signal. Afterproviding the first wire rate clock signal, in embodiments, the firstwire rate clock signal may be transmitted to the serializer.

Referring back to FIG. 13A, at a step 1306, an interim transmitter sideclock signal having a sixth frequency and a sixth phase is generated bythe serializer in the field programmable gate array. In embodiments, thesixth frequency corresponds to the third frequency. In embodiments thesixth phase may be aligned with the third phase. In embodiments, thesixth frequency may correspond to the third frequency and the sixthphase may not be aligned with the third phase. In embodiments, the sixthphase and the third phase may not be aligned. In embodiments, the thirdphase and the sixth phase may have a difference in phases that is lessthan a second threshold level.

At a step S1308, the interim transmitter side clock signal istransmitted from the serializer to the phase detector, which may or maynot be on the field programmable gate array. If the phase detector isnot on the field programmable gate array, the interim transmitter sideclock signal may be transmitted from the serializer to the phasedetector via a second clock output pin of the second interface. Inembodiments, the interim transmitter side clock signal may also betransmitted via a second zero delay buffer phase lock loop of the fieldprogrammable gate array before being transmitted via the second clockoutput in of the second interface.

The process of FIGS. 13A-13B may continue with at a step S1310.Referring to FIG. 13B, at a step S1310, a first output is generated bythe phase detector. The first output, in embodiments, is based at leaston a comparison of the first receiver side clock signal and the interimside clock signal. The first output, in embodiments, may includeinformation indicating the phase differences (if there is any) betweenthe first receiver side clock and the interim side clock signal.Additionally, in embodiments, the first output may include informationindicating a difference (if there is any) of frequencies between thefirst receiver side clock signal and the interim side clock signal.

At a step S1312, the first output may be transmitted from the phasedetector to a phase controller of the field programmable gate arraysystem, which may be placed within the field programmable gate array.

At a step S1314, interim adjustment information is determined by thephase controller. In embodiments, the interim adjustment information maybe based at least in part on the first output. For example, the interimadjustment information may indicate necessary adjustments to align thephases of the first receiver side clock signal and the interim sideclock signal. As another example, the interim adjustment information mayindicate necessary adjustments to align the frequencies of the firstreceiver side clock signal and the interim side clock signal.

At a step S1316, the interim adjustment information is transmitted fromthe phase controller to the phase lock loop with phase adjustment. Inembodiments, once the interim adjustment information is received, thephase lock loop with phase adjustment adjusts the second clock signalbased on the interim adjustment information. The adjustment of thesecond clock signal, in embodiments, may change the phase of the secondclock signal. In embodiments, the interim adjustment informationindicates a desired phase. In embodiments, the interim adjustmentinformation indicates a desired frequency. In embodiments, the interimadjustment information indicates a change in phase. In embodiments, theinterim adjustment information indicates a change in frequency. Inembodiments, the adjustment information includes a voltage. Inembodiments, the adjustment information includes a digital transmission.For example, the adjustment information may be a command over a serialbus (e.g., I2C or SPI) that adjusts a clock divider or phase rotator. Asanother example, the adjustment information may trigger a set ofparallel digital wires for “frequency/phase up,” “frequency/phase down,”and/or “frequency/phase step.” In embodiments, the adjustment of thesecond clock signal may change the frequency of the second clock signal.

In embodiments, steps S1302 through steps S1316 are repeated until thefirst output of the phase detector is below the first threshold level.This threshold level may indicate that the first receiver side clocksignal and the interim transmitter side clock signal have phases thatare aligned. This threshold level may indicate that the first receiverside clock signal and the interim transmitter side clock signal havefrequencies that correspond to one another.

Referring back to FIG. 12B, once the first output of the phase detectoris below a first threshold level, at a step S1222, the serializer maytransmit the first transmitter side clock signal to the computationalcircuitry. At a step S1224, a first set of operations performed by thecomputational circuitry on at least a portion of the first plurality ofdata items to generate a second plurality of data streams. Inembodiments, the first operation includes a trading algorithm. Inembodiments the trading algorithm may include the parsing the marketdata. Once the market data is parsed, mathematical operations areperformed at a portion of the market data. After the mathematicaloperations are performed, order packets are generated using at least anoutput of the mathematical operations. In embodiments, the first set ofoperations includes at least one of the following: (i) an arithmeticoperation; (ii) a logical operation; (iii) a pipeline operation; and(iv) a memory access operation.

In embodiments, at least a portion of the first set of operations isperformed prior to step S1222. In embodiments, at least a portion of thefirst set of operations is performed after step S922. In embodiments allof the first set of operations is performed after step S1222.

In embodiments, the second plurality of data streams may include asecond amount of data streams. In embodiments, the second amount of datastreams may be the same as the first amount of data streams. Inembodiments, the second amount of data streams is equal to eight (8)data streams. In embodiments, the second amount of data streams is equalto ten (10) data streams. In embodiments, the second amount of datastreams is equal to sixteen (16) data streams. In embodiments, thesecond amount of data streams is equal to twenty (20) data streams. Inembodiments, the second amount of data streams is equal to thirty-two(32) data streams. In embodiments, the second amount of data streams isequal to forty (40) data streams. In embodiments, the second amount ofdata streams is equal to sixty-four (64) data streams. In embodiments,the second amount of data streams is equal to eighty (80) data streams.In embodiments, the second amount of data streams is equal to onehundred twenty-eight (128) data streams. In embodiments, the secondamount of data streams is equal to one hundred sixty (160) data streams.In embodiments, the second amount of data streams is equal to two (2) tothe power of N, where N is an integer. In embodiments, the second amountof data streams is equal to ten (10) times two (2) to the power of N,where N is an integer. In embodiments the second amount of data streamsis equal to thirty-three (33) times two (2) to the power of N, where Nis an integer.

At a step S1226, the second plurality of data streams is transmittedfrom the computational circuitry to the serializer. Once received by theserializer, at a step S1228, the serializer converts the secondplurality of parallel data streams into the second serial data stream.The second serial data stream, in embodiments, includes trading data.Trading data, for example, may be any data related to purchasing orselling of stocks, commodities, goods, and/or services.

At a step S1230, the second serial data stream may be transmitted fromthe serializer off the field gate programmable array system via a secondplurality of data pins of the first interface. In embodiments, thesecond serial data stream is transmitted from the serializer on thefield programmable gate array to an input/output module of the fieldgate array system but not on the field gate array. The second serialdata stream may then be transmitted from the input/output module off thefield gate array system.

FIGS. 15A-C, 16A-B and 17 are exemplary flow charts of a processimplementing the field programmable gate array system in accordance withyet another embodiment of the present invention. In embodiments, theprocess described in the flow charts of FIGS. 15A-C, 16A-B and 17 may beimplemented by the field programmable gate array system shown in FIG. 3Bor FIG. 5B. The field programmable gate array system, including thefield programmable gate array and the components thereof, may be similarto the field programmable gate array systems of FIG. 3B and FIG. 5B, thedescriptions of which applying herein.

The process of FIGS. 15A-15C may begin at step S1502. Referring to FIG.15A, at step S1502, a first serial data stream is received by a firstplurality of data pins of a first interface of a field programmable gatearray in the field programmable gate array system. In embodiments, thefirst serial data stream may include market data. In embodiments, marketdata may be price and trade-related data for a financial instrumentreported by a trading venue (i.e., a stock exchange).

At a step S1504, a first clock signal having a first frequency and afirst phase is received by a first reference clock pin in the firstinterface from a fixed frequency oscillator. In embodiments, step S1504may be performed before step S1502. In embodiments, step S1504 may beperformed contemporaneously with step S1502.

The process of FIGS. 15A-15C may continue with a step S1506. At stepS1506, the first plurality of data pins of the first interface maytransmit the first serial data stream to a deserializer in the fieldprogrammable gate array.

At a step S1508, the first reference clock pin in the first interfacemay transmit the first clock signal to the deserializer. In embodiments,step S1508 may be performed before step S1506. In embodiments step S1508may be performed contemporaneously with step S1506.

At a step S1510, a first receiver side clock signal may be generated bythe deserializer. In embodiments, the first receiver side clock signalmay have a second frequency and a second phase. In embodiments, thesecond frequency is different from the first frequency. In embodiments,the second frequency corresponds to the first frequency. In embodiments,corresponding may refer to the frequencies being the same frequency. Inembodiments corresponding may also refer to frequencies being close(e.g., within an acceptable range in the art) to the same frequencies.In embodiments, the first phase and the second phase may not be aligned.In embodiments the first phase and the second phase are aligned. Inembodiments, a difference between the first phase and the second phaseis below a threshold level. In embodiments, the deserializer generatesthe first receiver clock signal based at least in part on the firstclock signal.

At a step S1512, the first serial data stream received by thedeserializer may be converted, by the deserializer, into a firstplurality of data streams. In embodiments, the first plurality of datastreams may include a first amount of data streams. In embodiments, thefirst amount of data streams is equal to eight (8) data streams. Inembodiments, the first amount of data streams is equal to ten (10) datastreams. In embodiments, the first amount of data streams is equal tosixteen (16) data streams. In embodiments, the first amount of datastreams is equal to twenty (20) data streams. In embodiments, the firstamount of data streams is equal to thirty-two (32) data streams. Inembodiments, the first amount of data streams is equal to forty (40)data streams. In embodiments, the first amount of data streams is equalto sixty-four (64) data streams. In embodiments, the first amount ofdata streams is equal to eighty (80) data streams. In embodiments, thefirst amount of data streams is equal to one hundred twenty-eight (128)data streams. In embodiments, the first amount of data streams is equalto one hundred sixty (160) data streams. In embodiments, the firstamount of data streams is equal to two (2) to the power of N, where N isan integer. In embodiments, the first amount of data streams is equal toten (10) times two (2) to the power of N, where N is an integer. Inembodiments the first amount of data streams is equal to thirty-three(33) times two (2) to the power of N, where N is an integer.

At a step S1514, the first receiver side clock signal is transmittedfrom the deserializer to computational circuitry in the fieldprogrammable gate array.

At a step S1516, the first plurality of data streams is transmitted formthe deserializer to the computational circuitry. In embodiments, thefirst plurality of data streams includes a first plurality of dataitems. In embodiments, step S1516 may be performed before step S1514. Inembodiments step S1516 may be performed contemporaneously with stepS1514.

At a step S1518, the first receiver side clock signal is transmittedfrom the deserializer to a phase detector of the field programmable gatearray system, which may or may not be one the field programmable gatearray. If the phase detector is not on the field programmable gatearray, the first receiver side clock signal may be transmitted from thedeserializer to the phase detector via a first clock output pin on asecond interface of the field programmable gate array. In embodiments,the first receiver side clock signal may also be transmitted via a firstzero delay buffer phase lock loop of the field programmable gate arraybefore being transmitted via the first clock output pin of the secondinterface.

At a step S1520, a first transmitter side clock signal is generated bythe field programmable gate array system. In embodiments, the firsttransmitter side clock signal has a third frequency and a third phase.In embodiments, the third frequency corresponds to the second frequency.In embodiments, the third phase may be aligned with the second phase. Inembodiments, the third phase and the second phase may not be aligned. Inembodiments, the third phase and the second phase may have a differencein phases that is less than a second threshold level. In embodiments,the first transmitter side clock signal is generated by performing thefollowing steps until a first output of the phase detector is below afirst threshold.

Referring to FIG. 16A, generating the first transmitter side clocksignal may begin at step S1602. At step S1602, an adjustable transceiverphase lock loop receives the first clock signal from the fixed frequencyoscillator via a second reference clock pin in the first interface. Inembodiments, the received first clock signal is used by the adjustabletransceiver phase lock loop to generate a second clock signal. Inembodiments, the second clock signal may have a fourth frequency and afourth phase.

The process of FIGS. 16A-B may continue at a step S1604. At step S1604,a first wire rate clock signal having a fifth phase and a fifthfrequency is generated. In embodiments, the generation of the first wirerate clock signal may be based in part on the second clock signal. Inembodiments the fifth frequency is a rational multiple of the fourthfrequency. In embodiments the fifth frequency is equal to x times thefifth frequency where x is the second amount of data streams. Inembodiments the fifth frequency is equal to x times the frequency of theof the transmitter side clock signal where x may be half of the secondamount of data streams.

In embodiments, the generation of the first wire rate clock signal maybe performed with the following steps. Referring to FIG. 17, at a stepS1702, the second clock signal is generated by the adjustabletransceiver phase lock loop based at least in part on the received firstclock signal. At a step S1704, the second clock signal is processed bythe adjustable transceiver phase lock loop based on the interimadjustment information received from the phase controller (see S1614 andS1616). As a step S1706, the first wire rate clock signal is provided bythe adjustable transceiver phase lock look. The providing of the firstwire rate clock signal, in embodiments, may be based at least in part onthe processing of the second clock signal. Once the first wire rateclock signal is provided, at a step S1708. the first wire rate clocksignal is transmitted by the adjustable transceiver phase lock loop to aserializer on the field programmable gate array. In embodiments, thefirst wire rate clock signal is generated by a phase lock loop. Inembodiments, the phase lock loop that generates the first wire rateclock signal is on field programmable gate array. In embodiments, thephase lock loop that generates the first wire rate clock signal is partof the field programmable gate array system and not part of the fieldprogrammable gate array.

In embodiments, the generation of the first wire rate clock signal mayinclude processing the second clock signal. Once the second clock signalis processed, the first wire rate clock signal may be provided. Theproviding of the first wire rate clock signal, in embodiments, is basedat least in part on the processing of the second clock signal. Afterproviding the first wire rate clock signal, in embodiments, the firstwire rate clock signal may be transmitted to the serializer.

Referring back to FIG. 16A, at a step 1606, an interim transmitter sideclock signal having a sixth frequency and a sixth phase is generated bythe serializer in the field programmable gate array. In embodiments, thesixth frequency corresponds to the third frequency. In embodiments thesixth phase may be aligned with the third phase. In embodiments, thesixth frequency may correspond to the third frequency and the sixthphase may not be aligned with the third phase. In embodiments, the sixthphase and the third phase may not be aligned. In embodiments, the thirdphase and the sixth phase may have a difference in phases that is lessthan a second threshold level.

At a step S1608, the interim transmitter side clock signal istransmitted from the serializer to the phase detector, which may or maynot be on the field programmable gate array. If the phase detector isnot on the field programmable gate array, the interim transmitter sideclock signal may be transmitted from the serializer to the phasedetector via a second clock output pin of the second interface. Inembodiments, the interim transmitter side clock signal may also betransmitted a second zero delay buffer phase lock loop of the fieldprogrammable gate array before being transmitted via the second clockoutput in of the second interface.

The process of FIGS. 16A-16B may continue with at a step S1610.Referring to FIG. 16B, at a step S1610, a first output is generated bythe phase detector. The first output, in embodiments, is based at leaston a comparison of the first receiver side clock signal and the interimside clock signal. The first output, in embodiments, may includeinformation indicating the phase differences (if there is any) betweenthe first receiver side clock and the interim side clock signal.Additionally, in embodiments, the first output may include informationindicating a difference (if there is any) of frequencies between thefirst receiver side clock signal and the interim side clock signal.

At a step S1612, the first output may be transmitted from the phasedetector to a phase controller of the field programmable gate arraysystem, which may be placed within the field programmable gate array.

At a step S1614, interim adjustment information is determined by thephase controller. In embodiments, the interim adjustment information maybe based at least in part on the first output. For example, the interimadjustment information may indicate necessary adjustments to align thephases of the first receiver side clock signal and the interim sideclock signal. As another example, the interim adjustment information mayindicate necessary adjustments to align the frequencies of the firstreceiver side clock signal and the interim side clock signal.

At a step S1616, the interim adjustment information is transmitted fromthe phase controller to the adjustable transceiver phase lock loop. Inembodiments, once the interim adjustment information is received, theadjustable transceiver phase lock loop adjusts the second clock signalbased on the interim adjustment information. The adjustment of thesecond clock signal, in embodiments, may change the phase of the secondclock signal. In embodiments, the interim adjustment informationindicates a desired phase. In embodiments, the interim adjustmentinformation indicates a desired frequency. In embodiments, the interimadjustment information indicates a change in phase. In embodiments, theinterim adjustment information indicates a change in frequency. Inembodiments, the adjustment information includes a voltage. Inembodiments, the adjustment information includes a digital transmission.For example, the adjustment information may be a command over a serialbus (e.g., I2C or SPI) that adjusts a clock divider or phase rotator. Asanother example, the adjustment information may trigger a set ofparallel digital wires for “frequency/phase up,” “frequency/phase down,”and/or “frequency/phase step.” In embodiments, the adjustment of thesecond clock signal may change the frequency of the second clock signal.

In embodiments, steps S1602 through steps S1016 are repeated until thefirst output of the phase detector is below the first threshold level.This threshold level may indicate that the first receiver side clocksignal and the interim transmitter side clock signal have phases thatare aligned. This threshold level may indicate that the first receiverside clock signal and the interim transmitter side clock signal havefrequencies that correspond to one another.

Referring back to FIG. 15B, once the first output of the phase detectoris below a first threshold level, at a step S1522, the serializer maytransmit the first transmitter side clock signal to the computationalcircuitry. At a step S1524, a first set of operations is performed bythe computational circuitry on at least a portion of the first pluralityof data items to generate a second plurality of data streams. Inembodiments, the first operation includes a trading algorithm. Inembodiments the trading algorithm may include the parsing the marketdata. Once the market data is parsed, mathematical operations areperformed at a portion of the market data. After the mathematicaloperations are performed, order packets are generated using at least anoutput of the mathematical operations. In embodiments, the first set ofoperations includes at least one of the following: (i) an arithmeticoperation; (ii) a logical operation; (iii) a pipeline operation; and(iv) a memory access operation.

In embodiments, at least a portion of the first set of operations isperformed prior to step S1522. In embodiments, at least a portion of thefirst set of operations is performed after step S1522. In embodiments,all of the first set of operations is performed after step S1522.

In embodiments, the second plurality of data streams may include asecond amount of data streams. In embodiments, the second amount of datastreams may be the same as the first amount of data streams. Inembodiments, the second amount of data streams is equal to eight (8)data streams. In embodiments, the second amount of data streams is equalto ten (10) data streams. In embodiments, the second amount of datastreams is equal to sixteen (16) data streams. In embodiments, thesecond amount of data streams is equal to twenty (20) data streams. Inembodiments, the second amount of data streams is equal to thirty-two(32) data streams. In embodiments, the second amount of data streams isequal to forty (40) data streams. In embodiments, the second amount ofdata streams is equal to sixty-four (64) data streams. In embodiments,the second amount of data streams is equal to eighty (80) data streams.In embodiments, the second amount of data streams is equal to onehundred twenty-eight (128) data streams. In embodiments, the secondamount of data streams is equal to one hundred sixty (160) data streams.In embodiments, the second amount of data streams is equal to two (2) tothe power of N, where N is an integer. In embodiments, the second amountof data streams is equal to ten (10) times two (2) to the power of N,where N is an integer. In embodiments the second amount of data streamsis equal to thirty-three (33) times two (2) to the power of N, where Nis an integer.

At a step S1526, the second plurality of data streams is transmittedfrom the computational circuitry to the serializer. Once received by theserializer, at a step S1528, the serializer converts the secondplurality of parallel data streams into the second serial data stream.The second serial data stream, in embodiments, includes trading data.Trading data, for example, may be any data related to purchasing orselling of stocks, commodities, goods, and/or services.

At a step S1530, the second serial data stream may be transmitted fromthe serializer off the field gate programmable array system via a secondplurality of data pins of the first interface. In embodiments, thesecond serial data stream is transmitted from the serializer on thefield programmable gate array to an input/output module of the fieldgate array system but not on the field gate array. The second serialdata stream may then be transmitted from the input/output module off thefield gate array system.

EXAMPLES

The following examples may be used to illustrate embodiments of thepresent invention. They are meant solely for illustration and notintended to be limiting.

Example 1

In embodiments a trading algorithm may be applied in a crossing auctionon an option exchange. In embodiments, a third serial data streamincluding target price information on a set of options contracts isprovided as in input to the FPGA. In embodiments, the first serialstream includes market data includes bid, asks, trades and auctionnotifications. When an auction is announced, as indicated in the marketdata, the FPGA compares the announced auction price to its target price.In embodiments, the target price is stored in the FPGA and is providedvia the third serial data stream. In embodiments, the target price maybe generated using a suitable pricing model, e.g., Black Scholes model.When there is some overlap, the field programmable gate array generatesand sends an order at its target price which is transmitted out of theFPGA in the second serial data stream. In embodiments, the target priceinformation on a set of options contracts is streamed into the fieldprogrammable gate array as a simple set of triggers in the third serialdata stream. The target price information, may include a linearizationof the result of the Black Scholes computation or some other model. Thefield programmable gate array may implement Black Scholes, and insteadreceive market data on the underlying a et as a secondary data stream.

Example 2

Another example of a trading algorithm is “signal based”. Inembodiments, signals refer to a trigger that is activated when certainevents happen. In embodiments, a signal may simply track a sale pricewhich changes every time the market information indicates a sale hastaken place. In embodiments, a signal may be used as a measure of bookpressure to determine whether prices are likely to rise or fall. Somesignals may be calculated quickly while others are calculated moreslowly because they require more data over a longer period of time. Inembodiments, order information may be based on a single signal or basedon a plurality of signals. Signals depending on the historical behaviorof a financial instrument are programmed into the field programmablegate array. A few simple and well-known examples of a signal here aremoving averages of past prices or book pressure signals (looking for animbalance in the order book). Signal-based trading involves receivingmarket data on the financial instrument, constructing the order book forthat instrument, calculating a predicted price based on a pre-definedcollection of signals, and sending an order if the signals indicate thatthe signal indicates that it is a favorable time to trade. A secondarydata stream here could be used to enable or disable signals or to changesignal parameter values. In embodiments, the secondary data stream maybe omitted. An example of a signal-based trading system might be asystem that updates the quotes for a market maker when a set of signalsindicates that a large directional move in the price of a stock iscoming. The parallel processing advantage shows up here more so than inthe first example.

Now that embodiments of the present invention have been shown anddescribed in detail, various modifications and improvements thereon canbecome readily apparent to those skilled in the art. Accordingly, theexemplary embodiments of the present invention, as set forth above, areintended to be illustrative, not limiting. The spirit and scope of thepresent invention is to be construed broadly.

What is claimed is:
 1. A field programmable gate array systemcomprising: (a) a field programmable gate array comprising: (1) a firstinterface comprising: (A) a first reference clock pin, wherein saidfirst reference clock pin is configured to receive a first clock signalhaving a first frequency and a first phase; (B) a second reference clockpin, wherein said second reference clock pin is configured to receivethe first clock signal; (C) a first plurality of data pins, wherein saidfirst plurality of data pins is configured to receive a first serialdata stream; (D) a second plurality of data pins, wherein said secondplurality of data pins is configured to transmit a second serial datastream; (2) a deserializer operationally connected to: (A) the firstreference clock pin to receive as a first input the first clock signaland (B) the first plurality of data pins to receive as a second inputthe first serial data stream, and wherein the deserializer is configuredto: (A) convert the first serial data stream into a first plurality ofparallel data streams having a first amount of data streams, and (B)generate a first receiver side clock signal based on the first clocksignal, wherein the first receiver side clock signal has a thirdfrequency and a third phase; and (C) transmit the first plurality ofparallel data streams and the first receiver side clock signal withinthe field programmable gate array; (3) computational circuitryoperationally connected to the deserializer to receive the firstplurality of parallel data streams and the first receiver side clocksignal, wherein the computational circuitry is configured to perform afirst set of operations on the first plurality of parallel data streamsto generate a second plurality of parallel processed data streams havinga second amount of data streams and the computational circuitry performsthe first set of operations to generate the second plurality of parallelprocessed data without use of a clock domain crossing circuit; (4) aserializer operationally connected to: (A) an adjustable transceiverphase lock loop to receive as a third input a first wire rate clocksignal, wherein the first wire rate clock signal has a fourth frequencyand a fourth phase; (B) the second plurality of data pins to transmit asa first output the second serial data stream; (C) the computationalcircuitry, wherein the serializer receives the second plurality ofparallel processed data streams from the computational circuitry and theserializer transmits to the computational circuitry a first transmitterside clock signal including a fifth frequency and a fifth phase; andwherein the serializer is configured to: (A) convert the secondplurality of parallel processed data streams into the second serial datastream; (B) generate the first transmitter side clock signal based onthe first wire rate signal, wherein the first transmitter side clocksignal has the fifth frequency and the fifth phase, wherein the fifthfrequency is different than and less than the fourth frequency; and (C)transmit the second serial data stream to the second plurality of datapins for transmission off the field programmable gate array; and (b) aphase control circuit, provided at least partially on the fieldprogrammable gate array, wherein the phase control circuit comprises:(1) a phase detector configured to compare the third phase of the firstreceiver side clock signal to the fifth phase of the transmitter sideclock signal and to generate a phase difference indicator signal basedon a difference between the third phase of the first receiver side clocksignal and the fifth phase of the transmitter side clock signal; (2) aphase controller operationally connected to the phase detector andconfigured to receive the phase difference indicator signal, and whereinthe phase controller is configured to determine adjustment informationbased on the phase difference indicator signal; and (3) the adjustabletransceiver phase lock loop operationally connected to the phasecontroller and configured to receive the adjustment information as wellas operationally connected to the second reference clock pin of thefirst interface of the field programmable gate array, wherein theadjustable transceiver phase lock loop is configured to generate asecond clock signal including the second frequency and the second phasebased on the first clock signal received via the second reference clockpin and process the second clock signal based on the adjustmentinformation received from the phase controller to generate the firstwire rate clock signal, wherein the transmitter side clock signal andthe first receiver side clock signal are phase aligned so that there isa fixed phase difference between the third phase and the fifth phase. 2.The system of claim 1, wherein the third frequency corresponds to thefirst frequency.
 3. The system of claim 2, wherein the third phase isnot aligned with the first phase.
 4. The system of claim 1, wherein thethird frequency is different than the first frequency.
 5. The system ofclaim 1, wherein the first set of operations does not include clockdomain crossing operations that delays processing of the first set ofparallel data streams.
 6. The system of claim 1, wherein the first setof operations includes at least one of the following: (i) an arithmeticoperation; (ii) a logical operation; (iii) a pipeline operation; and(iv) a memory access operation.
 7. The system of claim 1, wherein thefirst amount of data streams is the same as the second amount of datastreams.
 8. The system of claim 7, wherein the first amount of datastreams and the second amount of data streams are equal to one of thefollowing: (i) eight (8) data streams; (ii) ten (10) data streams; (iii)sixteen (16) data streams; (iv) twenty (20) data streams; (v) thirty-two(32) data streams; (vi) forty (40) data streams; (vii) sixty-four (64)data streams; (viii) eighty (80) data streams; (ix) one hundredtwenty-eight (128) data streams; and (x) one hundred sixty (160) datastreams.
 9. The system of claim 7, wherein the first amount of datastreams and the second amount of data streams are equal to two (2) tothe power of N, where N is an integer.
 10. The system of claim 7,wherein the first amount of data streams and the second amount of datastreams are equal to ten (10) times two (2) to the power of N, where Nis an integer.
 11. The system of claim 7, wherein the first amount ofdata streams and the second amount of data streams are equal tothirty-three (33) times two (2) to the power of N, where N is aninteger.
 12. The system of claim 1, wherein the fourth frequency isequal to x times the fifth frequency, where x is the second amount ofdata streams.
 13. The system of claim 1, wherein the fourth frequency isequal to x times the fifth frequency, where x is half of the secondamount of data streams.
 14. The system of claim 1, wherein the fourthfrequency is a rational multiple of the second frequency.
 15. The systemof claim 1, wherein the phase difference indicator signal is a pulsesignal.
 16. The system of claim 1, wherein adjustment informationindicates at least one of the following: (A) a desired phase; (B) adesired frequency; (C) a change in phase; and (D) a change in frequency.17. The system of claim 16, wherein adjustment information comprises avoltage.
 18. The system of claim 17, wherein adjustment informationcomprises a digital transmission.
 19. The system of claim 1, wherein theadjustment information is used to do at least one of the following: (i)set a bias to an oscillator; (ii) set a divider ratio; and (iii) set adelay.
 20. The system of claim 1, wherein the first serial data streamcomprises market data, the second serial data stream comprises orderentry data, and the first operation comprises a trading algorithm. 21.The system of claim 1, wherein the first serial data stream includesmarket data and the second serial data stream includes trading data. 22.The system of claim 1, wherein the field programmable gate array furthercomprises: (5) a second interface comprising: (A) a first clock outputpin configured to transmit the first receiver side clock signal, whereinthe first clock output pin is operationally connected to thedeserializer; and (B) a second clock output pin configured to transmitthe first transmitter side clock signal, wherein the second clock outputpin is operationally connected to the serializer; and wherein the phasedetector is not on the field programmable gate array and isoperationally connected to the first clock output pin and the secondclock output pin of the second interface of the field programmable gatearray.